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ABSTRACT 


The  U.  S.  Navy's  Integrated  Underwater  Surveillance  System  (IUSS)  monitors 
hydrophones  in  the  northeast  Pacific.  Geophysicists  studying  the  IUSS  data  find  seismic 
events  and  correlate  them  between  hydrophones  to  locate  their  sources.  Marine  biologists 
and  intelligence  personnel  are  interested  in  the  identification  and  localization  of  other 
sounds  in  the  IUSS  data.  The  current  means  of  identifying  and  correlating  sounds  is  a 
laborious  visual  examination  of  the  data  on  a  graphics  workstation. 

In  this  thesis,  a  computer-vision  method  is  presented  for  automatically  identifying  the 
sources  of  low-frequency  sounds  that  are  received  on  the  IUSS  hydrophones.  Also 
presented  in  this  thesis  is  a  blackboard  architecture  for  correlating  sound  "shapes"  between 
hydrophones  using  a  time-shift  transform.  The  methods  in  this  thesis  properly  identify 
approximately  90%  of  apparent  whale  moans  and  100%  of  seismic  events.  The  use  of  the 
time-shift  transform  has  resulted  in  nearly  a  100%  success  rate  in  correlating  whale  moans 
between  far-field  hydrophones,  despite  marked  sound  distortions  with  distance. 
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I.  INTRODUCTION 

A.    GOALS 

There  were  two  major  goals  of  this  thesis.  The  first  goal  was  to  develop  a  computer- 
vision  method  of  automatically  identifying  the  sources  for  discrete  low-frequency 
underwater  sounds.  These  sounds  are  received  by  fixed  hydrophones  located  off  of  the  west 
coast  of  the  United  States.  The  sounds  are  digitized  at  128  Hertz.  The  second  goal  was  to 
develop  computer-vision  techniques  for  automatically  correlating  the  shapes  of  these 
signals  to  closely  spaced  pairs  of  hydrophones  that  are  hundreds  of  kilometers  apart. 

Artificial-intelligence  vision-processing  techniques  were  used  to  achieve  the  first 
goal.  Approximately  90%  of  large  baleen  whale  moans  are  properly  identified,  nearly  all  t- 
phase  signals  from  earthquakes  are  identified,  and  man-made  calibration  signals  were  also 
identified.  The  shape  correlation  goal  was  achieved  through  the  use  of  an  artificial- 
intelligence  blackboard  architecture  and  by  computer-vision  correlation  of  regions  from 
two  sets  of  distantly  spaced  hydrophones.  A  comparison  of  this  method  against  the 
traditional  correlation  of  signals  using  Fast  Fourier  Transforms  shows  this  to  be  a  valid 
technique. 

There  were  two  motivations  for  this  thesis.  Current  processing  on  the  acoustic  signals 
attempts  to  automatically  detect  the  occurrence  of  t-phase  signals.  There  are  many  false 
identifications  of  t-phase  signals  that  are  actually  whale  moans.  This  thesis  addresses  that 
problem.  The  other  motivation  for  this  thesis  is  that  the  knowledge  about  blue  and  fin  whale 
migration  patterns  and  stock  assessments  is  very  limited.  This  thesis  demonstrates  that 
acoustic  tracking  of  large  baleen  whales  is  feasible. 

The  techniques  developed  for  this  thesis  can  be  used  to  build  a  system  that  will 
automatically  track  large  baleen  whales  and  to  automatically  detect  and  locate  submarine 
seismic  events. 


B.    ORGANIZATION 

Chapter  n  of  this  thesis  discusses  background  about  noises  that  are  produced 
underwater  and  how  these  noises  can  propagate  for  long  distances.  Chapter  II  also  reviews 
the  previous  work  that  has  been  done  with  t-phase  detection  and  correlation,  and  with  large 
baleen  whale  identification.  Chapter  III  discusses  the  methods  used  to  identify  discrete  low- 
frequency  underwater  signals.  Chapter  IV  discusses  the  methods  used  to  correlate  signals 
between  nearby  hydrophones  as  well  as  far-apart  hydrophones. 

Chapter  V  gives  a  description  of  the  computer  program  that  was  written  to  solve  the 
identification  problem.  The  identification  program  was  written  in  the  Prolog  language. 
Chapter  VI  describes  the  computer  program  that  was  written  to  solve  the  correlation 
problem.  The  correlation  program  was  written  with  the  Common  Lisp  Object  System 
(CLOS).  Chapter  VII  is  a  discussion  of  the  results  of  this  research  including  the 
performance  of  the  computer  programs.  Chapter  VIII  presents  the  conclusions  of  this 
thesis.  Both  achievements  and  weaknesses  are  discussed,  as  well  as  suggestions  for  further 
research. 


II.  SURVEY  OF  PREVIOUS  WORK 

A.  INTRODUCTION 

The  United  States  Navy  has  recently  declassified  many  aspects  of  its  undersea  acoustic 
surveillance  systems  (Fox,  1993).  These  systems  are  now  being  used  by  geophysicists  in 
the  VENTS  program,  the  sponsor  of  this  work,  to  monitor  undersea  earthquakes  and 
volcanoes.  The  undersea  surveillance  system  also  has  great  potential  for  use  in  tracking  and 
estimating  populations  of  marine  mammals,  including  the  largest  baleen  whales.  This 
chapter  discusses  background  concerning  the  use  of  the  undersea  surveillance  systems.  An 
overview  of  the  current  state  of  t-phase  processing  is  then  presented.  A  brief  description  of 
computer  vision  is  also  presented  along  with  its  relevance  to  the  processing  of  digitally 
recorded  hydrophones.  The  chapter  ends  with  a  discussion  of  the  research  on  large  baleen 
whale  acoustics. 

B.  BACKGROUND 

The  National  Oceanic  and  Atmospheric  Administration  (NOAA),  Pacific  Marine 
Environmental  Laboratory  (PMEL),  Ocean  Environment  Research  Division  (OERD)  is 
conducting  an  ongoing  study  of  the  impact  that  the  dynamics  of  the  geological  processes 
occurring  on  the  mid-ocean  spreading  ridges  have  on  the  Earth's  oceans.  Earthquakes  and 
volcanism  play  a  major  part  in  these  dynamic  processes.  This  ongoing  study  is  known  as 
the  VENTS  program. 

The  United  States  Navy  has  for  many  years  had  an  undersea  surveillance  system  that 
has  been  used  for  monitoring  submarine  activities  (Fox,  1993).  This  system  consists  of  both 
fixed  and  mobile  hydrophone  systems.  The  system  of  interest  in  this  thesis  is  the  SOund 
SUrveillance  System  (SOSUS).  SOSUS  consists  of  fixed  arrays  of  hydrophones  mounted 
at  the  approximate  depth  of  the  deep  sound  channel.  These  hydrophones  are  very  sensitive 
and  have  the  ability  to  pick  up  very  low  frequency  sounds. 


When  an  earthquake  occurs,  there  are  three  kinds  of  energy,  or  phases,  associated  with 
it.  First,  there  is  a  primary  phase  known  as  a  p-phase.  This  phase  consists  of  energy  that  is 
transmitted  as  compressional  shock  waves  within  the  earth's  crust.  The  secondary  or  s- 
phase  consists  of  transverse  shock  waves  which  are  also  transmitted  through  the  earth's 
crust.  The  tertiary  phase  of  an  earthquake,  or  t-phase,  is  only  associated  with  underwater 
earthquakes.  The  t-phase  is  acoustic  energy  which  is  transmitted  into  the  water  column 
from  the  shaking  of  the  ocean  floor.  The  definition  of  t-phase  has  been  expanded  to  include 
any  low-frequency  seismic  event  which  transmits  acoustic  energy  into  the  water  column 
(Hammond,  1990).  T-phase  signals  which  are  received  at  remote  hydrophones  typically 
have  a  frequency  range  from  just  a  few  Hertz  to  just  a  few  tens  of  Hertz  (Hammond,  1990). 

There  are  two  qualities  of  the  physics  of  the  ocean  that  allow  us  to  study  t-phases  and 
other  low-frequency  acoustic  phenomena  at  long  ranges.  One  quality  is  that  the  attenuation 
of  sound  by  seawater  is  very  small  for  such  low  frequencies.  The  attenuation  rate  due  to 
absorption  in  seawater  varies  with  the  square  of  the  frequency.  For  a  frequency  of  100  Hz, 

the  attenuation  rate  due  to  absorption  is  approximately  10  decibels  per  meter  (Clay,  1977, 
pg.  100).  The  second  quality  is  the  deep  sound  channel  which  exists  in  all  of  the  worlds 
oceans.  This  deep  sound  channel  traps  sound  which  enters  it  at  a  small  enough  angle  and 
propagates  it  for  very  long  ranges  without  the  sound  interacting  with  either  the  ocean  floor 
or  the  ocean  surface.  This  reduces  the  attenuation  of  sound  due  to  spreading  from  a 
spherical  relationship  to  a  cylindrical  relationship. 

The  speed  of  sound  increases  with  increasing  pressure  and  it  decreases  with  decreasing 
temperature.  A  greater  water  depth  correlates  with  a  greater  pressure  at  that  depth.  Also,  a 
greater  water  depth  generally  correlates  with  a  colder  water  temperature  until  the 
temperature  reaches  a  few  degrees  above  freezing.  The  temperature  of  the  water  in  the 
ocean  at  depth  does  vary,  but  minimally.  There  are  many  components  to  the  sound  speed 
equation  for  seawater,  including  salinity,  but  the  factors  of  pressure  and  temperature  are 
primarily  responsible  for  creating  a  sound  speed  minimum  at  a  moderate  depth  in  the  ocean. 
The  depth  of  the  sound-speed  minimum  varies  depending  on  the  qualities  of  the  water 


column,  but  it  is  typically  around  800  meters  in  mid-latitudes.  Sound  traveling  in  the  water 
is  refracted  toward  areas  that  have  lower  sound  speeds.  This  sound  speed  minimum  is  the 
reason  that  the  deep  sound  channel  exists. 

C.    T-PHASE  PROCESSING 

One  of  the  goals  of  this  thesis  was  to  develop  a  technique  which  would  automatically 
identify  t-phases.  Loud  whale  moans  are  sometimes  mistakenly  identified  as  t-phases  and 
some  low-volume  t-phases  are  not  detected.  A  discussion  of  the  t-phase  processing  follows. 

In  1991  the  VENTS  program  installed  a  data  collection  system  at  a  Navy  facility  to 
digitize  and  record  data  from  the  SOSUS  hydrophones.  NOAA/PMEL/OERD  was  given 
access  to  hydrophones  from  five  hydrophone  arrays  scattered  around  the  northeast  Pacific 
ocean  basin.  Two  hydrophones  from  each  of  five  arrays,  plus  five  directed  beams,  are  being 
monitored  continuously  by  the  NOAA  system.  The  hydrophone  signals  are  digitized  at  a 
rate  of  128  Hertz  with  8  or  16  bit  accuracy,  depending  on  the  hydrophone.  The  digitized 
hydrophone  signals  are  recorded  to  2.3-Gigabyte  8-mm  tapes.  Approximately  one  tape  of 
data  per  week  is  collected  (Fox,  1992). 

The  collected  data  is  analyzed  by  scientists  at  OERD  using  signal  processing 
methodologies.  Fast  Fourier  Transforms  are  run  on  the  digitized  recordings  from  each 
hydrophone  to  transform  the  data  from  its  time  vs.  amplitude  raw  format  to  a  time  vs. 
frequency  format.  This  time-frequency  data  is  scrolled  across  a  computer  screen  while  a 
scientist  visually  looks  for  t-phases.  The  analysis  software  also  has  an  automatic  t-phase 
detector  built  in,  but  it  has  not  proved  trustworthy  enough  to  allow  the  data  tapes  to  be 
scanned  entirely  by  machine.  After  t-phase  events  are  identified,  they  are  correlated 
between  hydrophones  on  the  same  hydrophone  array.  This  correlation  gives  a  time 
difference  between  nearby  hydrophones.  From  this  information,  and  with  the  information 
provided  by  using  an  underwater  sound  propagation  model,  a  tentative  hyperbolic  line  of 
position  can  be  determined.  The  line  of  position  is  tentative  because  the  correlations  are 


sometimes  ambiguous.  The  best  six  correlations  are  saved  in  case  the  first  correlation 
proves  to  be  erroneous  (Fox,  1992). 

A  rough  location  can  be  determined  by  intersecting  the  hyperbolic  lines  of  position  on 
a  seismic  event  from  three  or  more  hydrophone  arrays.  The  position  obtained  by  using  time 
differences  of  the  closely  spaced  hydrophones  is  not  very  accurate  because  of  the  errors  that 
are  induced  by  the  proximity  of  the  hydrophones  combined  with  the  digitization  rate  of  the 
signal.  Once  a  rough  position  is  obtained,  a  more  accurate  position  can  be  determined  by 
time-correlating  the  t-phase  receptions  between  the  widely  spaced  hydrophones  of  the 
different  arrays  (Fox,  1992). 

D.    COMPUTER  VISION 

For  the  t-phase  processing,  when  the  time-frequency  data  from  the  hydrophones  is 
scrolled  across  the  computer  screen,  the  operator  identifies  the  t-phases  or  other  events  of 
interest  visually.  T-phases  have  shapes  which  are  easily  distinguished  by  a  human  operator 
looking  at  the  time-frequency  plot.  There  are  also  whale  moans  in  this  data,  and  they  have 
easily  distinguishable  shapes  that  are  visually  distinct  from  the  t-phases.  The  signal 
processing  techniques  that  are  used  in  the  analysis  of  these  sounds  are  not  designed  to 
distinguish  between  the  shapes  of  whale  moans  and  the  shapes  of  t-phases.  The  automatic 
detector  that  is  in  use  is  amplitude  based,  and  it  sometimes  provides  false  detections  due  to 
whale  moans  or  other  noises  (Fox,  1992). 

Vision  processing  techniques  are  well  suited  for  the  t-phase  problem.  Time-frequency 
plots  of  the  hydrophone  data  are  digitized  pictures.  The  t-phases  in  these  plots  are  easily 
recognized  by  a  human  operator.  They  have  simple  parameters  and  can  also  be  easily 
recognized  by  vision  processing  techniques.  The  whale  moans  are  more  complex,  but  they 
are  also  easily  identified  by  a  human  operator.  The  parameters  of  whale  moans  are  also 
more  difficult  to  identify,  but  most  whale  moans  can  be  recognized  by  vision  processing 
techniques.  A  short  discussion  of  computer  vision  processing  follows. 


Computer  vision  is  the  sub-field  of  artificial  intelligence  that  is  concerned  with 
emulating  human  visual  perception.  The  basic  problem  of  computer  vision  is  to  identify  the 
shapes  that  are  in  a  digitized  picture  (Charniak,  1985,  pg.  89).  Vision  processing  is  broken 
up  into  two  distinct  parts.  Low- level  processing  is  performed  on  the  entire  picture  in  order 
to  gain  information  from  the  raw  image  and  classify  it  for  later  processing.  The  results  of 
this  low-level  processing  is  typically  a  set  of  edges,  comers,  and  regions.  The  high-level 
processing  uses  the  knowledge  gained  by  the  low-level  processing  to  perform  a  cognitive 
analysis  of  the  picture  (Ballard,  1982,  pg.  6).  Usually,  this  high-level  processing  uses 
mathematical  descriptions  of  typical  shapes  as  its  knowledge  domain  in  order  to  identify 
the  shapes  in  the  picture  (Charniak,  1985,  pg.  150). 

Low-level  vision  processing  is  centered  around  the  concept  of  connected  pixels.  If  a 
picture  is  made  up  of  pixels  that  are  either  turned  on  or  turned  off,  then  strongly  connected 
pixels  are  pixels  that  are  turned  on  and  are  adjacent  to  each  other,  either  vertically  or 
horizontally.  Weakly  connected  pixels  are  pixels  that  are  turned  on  and  have  an  adjacent 
corner.  Disconnected  pixels  are  those  pixels  that  are  turned  on  and  are  neither  strongly  nor 
weakly  connected  (Dougherty,  1992). 

This  thesis  directly  uses  some  vision  processing  programs  by  Professor  Neil  C.  Rowe 
of  the  Naval  Postgraduate  School.  These  programs  perform  the  low-level  vision  processing 
needed  to  create  a  black-and-white  image  from  a  multi-spectral  image  and  they  also 
identify  regions  for  further  processing. 

E.    WHALE  RESEARCH 

During  the  course  of  the  research  for  a  thesis  on  t-phase  detection  and  correlation,  it 
became  apparent  to  us  that  there  might  be  some  benefit  from  this  thesis  to  the  community 
of  whale  researchers.  There  are  many  sounds  on  the  data  tapes  that  we  examined  that  are 
from  some  large  species  of  whale.  If  the  species  of  whale  can  be  determined,  and  if  an 
average  rate  of  moaning  can  be  determined,  then  the  techniques  presented  in  this  thesis  can 


be  used  to  help  determine  stock  assessments,  locations,  and  migration  patterns  of  these 
whales. 

The  whale  moans  that  occur  repeatedly  in  this  data  are  not  a  known  call  from  any 
species  of  whale  (McDonald,  1993).  There  has  been  speculation  that  these  moans  come 
from  either  blue  whales  (Balaenoptera  musculus)  or  fin  whales  {Balaenoptera  physalus) 
(Fox,  1993b).  These  two  species  of  whale  are  the  largest  living  species  of  animals 
(Mizroch,  1984a;  Mizroch,  1984b),  and  both  species  have  been  previously  associated  with 
low-frequency  moans,  although  these  moans  have  had  different  shape  and  duration  (Bird, 
1987;  Cummings  1971;  Cummings,  1986).  The  inability  of  researchers  to  associate  the 
moans  that  are  seen  in  the  t-phase  data  with  a  particular  species  indicates  the  inherently 
difficult  nature  of  studying  large  whales  that  live  in  the  open  ocean. 

Both  fin  whales  and  blue  whales  are  either  solitary  or  are  found  in  small  groups  and 
both  species  also  migrate  north-south  over  vast  stretches  of  ocean  each  year  (Mizroch, 
1984a;  Mizroch,  1984b).  The  stock  assessment  of  each  of  these  species  of  whale  is  very 
rough  due  to  the  difficulties  of  counting  solitary  animals  in  the  open  ocean  (Mizroch, 
1984a;  Mizroch,  1984b).  As  of  1984,  the  stock  estimates  of  the  blue  whale  in  the  north 
Pacific  indicate  that  it  has  not  made  any  recovery  from  the  time  that  the  killing  of  blue 
whales  by  man  ceased  in  1965  (Mizroch,  1984a). 


III.  IDENTIFICATION  OF  DISCRETE  SIGNALS 

A.  INTRODUCTION 

The  objective  of  the  signal  identification  part  of  this  thesis  was  to  identify  whale  moans 
and  t-phase  signals.  Complicating  this  objective  is  the  environment  within  which  the 
hydrophones  exist.  On  the  data  segments  that  we  analyzed,  there  were  some  time  periods 
that  were  extremely  noisy,  and  also  some  time  periods  where  the  hydrophone  signal  was 
replaced  with  a  calibration  signal.  There  was  some  noise  present  for  the  entire  data  set.  This 
chapter  discusses  the  methodology  used  to  identify  regions  from  a  noisy  data  set. 

B.  AUTOMATIC  IDENTIFICATION  OF  ACOUSTIC  SIGNALS 

Previous  efforts  to  automatically  identify  t-phase  signals  have  been  unreliable  because 
they  have  been  based  on  the  amplitude  of  the  incoming  signal  in  a  selected  frequency  band. 
Anything  that  makes  a  loud  enough  noise  in  the  same  band  will  be  detected  as  a  t-phase. 
Also,  any  earthquake  that  does  not  make  a  loud  enough  noise  will  not  be  detected  as  a  t- 
phase.  An  amplitude  detector  is  the  best  method  to  use  with  traditional  signal  processing 
methods,  even  with  the  limitations,  due  to  the  nature  of  these  methods. 

We  have  taken  a  different  approach  to  automatically  identifying  t-phase  signals.  This 
approach  also  has  the  side-benefit  of  identifying  most  other  signals  that  are  received  on  the 
hydrophone.  By  looking  at  the  time-frequency  plot  of  the  hydrophone  data  as  a  picture,  we 
use  computer-vision  processing  methods  to  identify  the  signals  that  are  digitized  from  the 
hydrophone.  This  emulates  the  process  of  a  human  operator  looking  at  the  data  and 
identifying  signals  of  interest. 

Figure  1  shows  a  time-voltage  plot  of  some  sounds  received  by  a  hydrophone  and  the 
transform  of  that  signal  into  the  time-frequency  domain.  Figure  1(a)  is  a  plot  of  the  raw 
signal  voltages  over  time.  There  is  one  minute  of  signal  reception  plotted,  or  7680  points 
of  raw  data.  Figure  1  (b),  (c),  and  (d)  show  this  same  signal  after  it  has  been  Fourier 
transformed  into  the  time-frequency  domain.  Transforms  were  performed  on  each  second 


of  data,  giving  a  one  second  resolution  in  the  time  scale  and  a  one  Hertz  resolution  in  the 
frequency  scale.  Only  one  side  of  the  symmetric  spectrum  from  0  to  64  Hertz  are  shown. 
The  black  pixels  in  Figure  1(b)  are  all  of  the  pixels  in  the  Fourier  transform  that  had  a 
magnitude  greater  than  15.  In  Figure  1(c),  the  black  pixels  are  those  with  a  magnitude 
greater  than  30.  In  Figure  1(d),  the  black  pixels  are  those  with  a  magnitude  greater  than  45. 
Each  element  of  a  Fourier  transform  array  represents  a  magnitude  of  sound  over  a  range  of 
frequencies.  For  these  Fourier  transform  arrays,  each  element  of  each  Fourier  transform 
array  represents  the  magnitude  of  the  sound  intensity  received  for  one  second  over  a  range 
of  one  Hertz. 
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Figure  1:  Time-voltage  and  time-frequency  plots  of  a  signal. 

The  geophysicists  that  are  monitoring  the  hydrophone  signals  for  t-phases  are  not 
burdened  with  the  problem  of  representing  magnitudes  in  black  and  white.  The  processing 
is  being  done  on  color  graphics  workstations.  Magnitudes  are  represented  by  an  array  of 
colors.  The  analyst  working  with  the  data  can  immediately  see  the  intensity  of  the  sound 
over  various  frequencies  by  the  color-coded  output. 

T-phases,  whale  moans,  and  other  identifiable  signals  have  distinct  shapes  that  can  be 
recognized  by  both  a  human  and  a  computer.  In  order  for  a  computer  to  recognize  a  signal, 
the  signal  must  first  go  through  some  low-level  vision-processing  which  will  take  the  time- 
frequency  data  and  find  cohesive  regions  while  eliminating  a  majority  of  the  noise.  We 
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have  developed  a  ten-step  low-level  processing  strategy  that  takes  a  set  of  Fast  Fourier 
Transform  magnitudes,  arranged  by  time  and  frequency,  and  creates  a  set  of  well-defined 
regions  that  can  be  identified  by  number. 

Our  low-level  vision  processing  creates,  in  essence,  a  black  and  white  (or  on  and  off) 
pixel  image  from  the  Fast  Fourier  Transform  magnitudes.  Noise  is  removed  from  this 
image  and  then  regions  have  gaps  filled  in.  Man-made  pixels  are  then  marked  and  removed. 
Man-made  regions  come  from  such  things  as  boat  noises  and  calibration  signals.  Once  all 
of  this  is  done,  regions  are  clumped  together.  The  final  low-level  step  is  to  associate  the 
turned-on  pixels  with  the  magnitudes  that  existed  in  the  original  data.  Chapter  V,  Section 
D  provides  details  on  these  steps. 

Once  the  low-level  processing  is  completed,  high-level  vision  processing  performs  the 
feature  extraction  from  the  identified  regions.  This  high-level  processing  groups  nearby 
regions  together  into  larger  logical  regions  and  determines  whether  they  are  t-phase  signals, 
whale  moans,  or  of  unknown  origin.  This  is  done  in  a  twelve-step  process  which  is 
described  in  detail  in  Chapter  V,  Section  E. 

C.    CHARACTERISTICS  OF  SIGNALS 

The  calibration  signal,  which  is  removed  in  the  low-level  processing,  always  has  the 
pixels  at  27  and  28  Hertz  turned  on.  It  also  always  has  the  pixels  between  the  frequencies 
of  3  and  24  Hertz  turned  off  and  the  pixels  between  the  frequencies  of  30  and  34  Hertz 
turned  off.  Other  pixels  may  or  may  not  be  turned  on,  but  this  description  is  adequate  for 
uniquely  describing  the  calibration  signal. 

Figure  2  shows  a  typical  calibration  signal.  In  the  first  minute  of  this  plot,  there  is  a 
whale  signal  and  some  random  noise.  The  calibration  signal  runs  for  approximately  two 
minutes.  During  that  two  minutes,  the  reception  of  the  hydrophone  signal  is  cut  off.  After 
the  calibration  signal  stops  at  about  the  third  minute  in  the  plot,  the  hydrophone  signal 
becomes  extremely  noisy.  For  this  figure  all  Fourier  transform  magnitudes  that  were 
greater  than  45  were  plotted. 
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Figure  2:  Time-frequency  plot  of  a  calibration  signal. 

Through  an  examination  of  the  Fourier  transformed  sounds,  we  found  that  whale 
moans  meet  the  following  minimum  criteria:  a  minimum  frequency  greater  than  20  Hz,  a 
time  span  between  12  and  60  seconds,  an  area  between  50  and  500  pixels,  and  a  density 
between  8  and  75%.  This  very  general  set  of  criteria  has  been  determined  to  be  good  enough 
to  identify  a  majority  of  the  whale  signals  without  being  so  general  as  to  mistakenly  identify 
other  signals  as  whales. 

Figure  3  shows  eight  complete  whale  moans  in  the  center  of  the  time-frequency  graphs 
and  parts  of  two  more  whale  moans  at  the  beginning  and  ending  times  in  the  graph.  The 
shapes  are  easily  identifiable  because  human  visual  processing  can  fill  in  disconnected 
regions.  Each  of  the  whale  moans  has  several  properties  in  common,  but  each  moan  is  also 
a  distinctly  different  sound.  For  this  figure,  all  Fourier  transform  magnitudes  that  were 
greater  than  35  were  plotted. 
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Figure  3:  Time-frequency  plot  of  several  whale  moans. 

Every  t-phase  signal  that  we  have  observed  has  a  minimum  frequency  of  less  than  10 
Hertz.  Many  t-phase  signals  have  the  additional  characteristic  that  their  maximum 
frequency  was  less  than  20  Hertz.  These  were  subdivided  into  a  class  called  far-field  t- 
phases  since  the  higher  frequencies  were  probably  frequency  attenuated. 

Figure  4  shows  a  t-phase  signal  in  addition  to  whale  moans.  The  t-phase  begins  at 
about  the  half  minute  point  in  the  plot  and  slowly  dies  out  by  the  fourth  minute.  There  are 
also  10  complete  and  two  partial  whale  moans  during  this  time.  For  this  figure,  all  Fourier 
transform  magnitudes  that  were  greater  than  35  were  plotted. 
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Figure  4:  Time-frequency  plot  of  a  t-phase  signal. 
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All  pixel  regions  which  did  not  meet  any  of  these  criteria,  were  classified  as 
unidentified  signals.  Some  of  these  unidentified  signals  were  actually  whale  moans,  but 
many  of  them  were  just  unidentifiable  noisy  signals. 

Figure  5  shows  a  particularly  noisy  time  period.  There  are  no  discernible  shapes  in  this 
signal.  The  cause  of  this  noise  could  possibly  be  a  defective  link  between  the  hydrophone 
and  the  data  collection  equipment.  We  speculate  that  this  is  the  cause  because  the 
calibration  signal  shows  up  frequently  around  the  noisy  areas  of  the  signal.  Extremely  noisy 
periods  also  occur  when  there  is  vessel  traffic  very  near  the  hydrophones.  For  this  figure, 
only  Fourier  transform  magnitudes  that  were  greater  than  130  were  plotted.  Nearly  every 
pixel  in  this  graph  has  a  magnitude  greater  than  45. 
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Figure  5:  Time-frequency  plot  of  a  noisy  signal. 
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IV.  CORRELATION  OF  SIGNALS 

A.    INTRODUCTION 

When  an  underwater  sound  is  correlated  between  two  hydrophones,  the  relative  time- 
shift  between  the  receptions  of  that  sound  can  be  determined.  Once  the  time  difference  of 
a  signal  reception  is  known,  a  hyperbolic  line-of-position  can  be  found.  Deterministic 
models  exist  for  calculating  lines-of-position,  given  the  transmission  qualities  of  the 
medium,  the  geometry  of  the  hydrophone  placements,  and  the  time  difference  of  the  signal 
reception  (Loje,  1983). 

Four  hydrophone  recordings  were  studied  for  this  thesis.  The  hydrophones  are  in  a 
geometry  of  two  pairs  of  hydrophones  that  are  far  apart.  The  hydrophones  in  each  pair  are 
approximately  400  meters  apart  and  the  pairs  of  hydrophones  are  approximately  340 
kilometers  apart  (Dziak,  1993).  Assuming  a  sound  speed  of  1500  meters  per  second  in  the 
deep  sound  channel,  the  travel  time  for  sound  between  the  nearby  hydrophones  is  about  a 
third  of  a  second,  and  the  travel  time  for  sound  between  the  far-apart  hydrophones  is  nearly 
4  minutes. 

The  objective  of  the  signal  correlation  part  of  this  thesis  was  to  correlate  identified 
sounds  between  the  nearby  hydrophones  and  then  to  correlate  these  nearby-correlated 
signals  between  the  far-apart  pairs  of  hydrophones.  There  are  two  different  problems 
presented  when  attempting  to  correlate  a  signal  between  the  nearby  hydrophones  and 
between  the  far-apart  hydrophones.  When  correlating  a  signal  between  nearby 
hydrophones  the  time  difference  is  short,  but  the  differences  in  hydrophone  characteristics, 
the  near-field  effects  of  the  environment,  and  the  effects  of  region  growing  in  the  signal 
identification  program  will  make  the  signals  look  different.  When  correlating  a  signal 
between  far  apart  hydrophones,  attenuation  and  time  stretching  will  have  additional  effects 
on  the  signal  shape  (Urick,  1975).  There  are  also  many  more  possible  signals  from  which 
to  choose  for  a  correlation. 
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B.    NEAR-FIELD  CORRELATIONS 

The  geophysicists  that  are  monitoring  the  SOSUS  hydrophones  find  the  time- 
differences  between  the  arrivals  of  signals  at  nearby  hydrophones  by  using  Fast  Fourier 
Transform  correlation  techniques.  The  purpose  of  this  nearby  hydrophone  sound 
correlation  is  to  determine  tentative  a  tentative  bearing  to  the  source  of  a  signal.  The 
preliminary  bearings  are  calculated  in  order  to  reduce  the  number  of  calculations  that  are 
needed  to  correlate  signals  to  the  far-apart  hydrophones.  "The  results  are  often  ambiguous 
and  a  selection  of  six  possible  azimuths  (and  back-azimuths)  are  recorded"  (Fox,  1992). 

The  purpose  of  correlating  signals  to  the  nearby  hydrophones  in  this  thesis  was  to 
determine  possible  candidates  for  correlation  to  the  far-apart  hydrophones.  Finding 
tentative  bearings  from  the  time  offsets  was  not  possible  using  our  approach  because  of  the 
resolution  of  the  shapes  created  in  the  identification  program  was  too  rough  relative  to  the 
travel  time  of  sound  between  the  nearby  hydrophones.  The  research  in  this  part  of  the  thesis 
concentrated  primarily  on  the  whale  moans.  The  whale  moans  are  much  more  frequent  than 
the  t-phase  signals  and  thus  present  a  better  opportunity  to  apply  artificial-intelligence 
techniques  to  their  correlation. 

An  artificial-intelligence  computing  model  known  as  the  blackboard  architecture  was 
applied  to  the  problem  of  correlating  signals  to  nearby  hydrophones.  The  blackboard 
architecture  was  appropriate  for  this  application  because  there  is  a  hierarchy  of  states  to 
which  the  regions  can  belong,  and  there  are  independent  knowledge  sources  that  can 
change  the  state  of  the  blackboard  based  on  the  facts  which  have  been  posted  to  the 
blackboard. 

The  framework  for  a  blackboard  consists  of  three  elements:  the  blackboard,  multiple 
knowledge  sources,  and  a  controller.  The  blackboard  consists  of  facts  that  are  arranged  in 
a  hierarchical  structure.  The  knowledge  sources  act  independently  on  the  facts  contained  in 
the  blackboard  and  can  add  new  facts  and  retract  old  facts.  The  controller  determines  when 
a  knowledge  source  should  be  invoked  (Nii,  1986). 
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The  structure  of  the  blackboard  for  matching  the  nearby  hydrophones  consisted  of: 
unmatched  regions,  tentatively  matched  regions,  definitely  matched  regions,  and 
unmatchable  regions.  The  characteristics  of  each  hydrophone  were  also  a  part  of  the 
blackboard  structure,  along  with  the  times  associated  with  the  matched  regions. 

The  knowledge  sources  for  the  correlation  program  were:  current-time,  time-span, 
time-separation,  best-match,  unmatchable-signal,  and  multiple-match.  Each  of  these 
knowledge  sources  contributed  information  to  the  blackboard,  and  when  new  knowledge 
was  added  to  the  blackboard,  the  knowledge  sources  were  able  to  act  on  it.  Some 
knowledge  sources  are  more  complex  than  others.  The  current-time  knowledge  source  just 
adds  the  start  time  of  the  most  recently  read  region  to  the  blackboard  while  the 
unmatchable-signal  knowledge  source  tests  the  current  time  against  the  start  times  of  all  of 
the  regions  in  the  unmatched-region  list  and  moves  regions  to  the  unmatchable-region  list 
based  on  the  maximum  possible  travel  time  between  the  hydrophones. 

The  controller  got  a  new  region  for  a  hydrophone  and  implemented  the  appropriate 
knowledge  sources.  The  new  region  would  either  be  added  to  the  unmatched-region  list  or 
the  tentatively-matched-region  list.  Once  this  was  done,  other  knowledge  sources  would  be 
called  on  to  move  unmatched  regions  to  the  unmatchable-region  list  and  to  move 
tentatively-matched  regions  to  the  definite-matched-region  list. 

Signals  were  matched  by  time  shifting  one  signal  so  that  its  starting  time  matched  the 
start  time  of  the  other  signal.  Then  the  number  of  pixels  that  the  regions  had  in  common 
was  found.  Doubling  the  number  of  overlapping  pixels  and  dividing  this  by  the  sum  of  the 
pixels  in  both  regions  produced  a  number  between  0  and  1  that  represented  a  goodness-of- 
fit  for  the  two  signals.  Each  signal  was  time-shifted  five  times  to  cover  a  time  period  of  two 
seconds  on  either  side  of  the  starting  times  of  the  signals.  The  maximum  goodness-of-fit 
was  taken  from  each  of  these  time  shifts.  Multiple  time  shifts  were  needed  because 
distortions  of  these  signals  during  their  transformations  into  shapes. 
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C.    FAR-FIELD  CORRELATIONS 

The  geophysicists  that  are  monitoring  the  SOSUS  hydrophones  find  tentative  bearings 
from  the  nearby  hydrophones  and  use  these  bearings  to  search  the  data  of  the  far-apart 
hydrophones  for  possible  correlations  of  events.  This  can  sometimes  result  in  signals  being 
incorrectly  matched  and  it  can  also  result  in  signals  not  being  matched  at  all  when  they 
should  be  matched. 

We  took  a  different  approach  in  this  thesis.  We  narrowed  down  the  total  number  of 
regions  that  could  be  matched  by  the  approximate  direct-line  travel  time  of  sound  from  one 
hydrophone  pair  to  the  other.  In  this  case,  the  total  travel  time  was  approximately  250 
seconds  which  includes  an  error  factor  for  not  knowing  the  transmission  characteristics  of 
the  water  column.  For  any  particular  sound  received  on  one  hydrophone  pair,  without  a- 
priori  knowledge  about  which  direction  that  sound  was  coming  from,  there  is  a  window  of 
approximately  500  seconds  of  sounds  to  check  against  from  the  other  hydrophone  pair.  For 
the  whale  moans  in  the  data  set  that  we  examined,  there  were  usually  15  to  20  moans  that 
needed  to  be  tested  from  one  hydrophone  pair  for  each  moan  recorded  on  the  other  pair. 

Even  though  time  stretching  and  attenuation  affect  a  signal  traveling  over  such  a  long 
distance,  we  found  that  using  the  same  time-shift  and  overlap  technique  that  was  applied  to 
the  correlation  of  signals  to  the  nearby  hydrophones  was  sufficient  to  match  signals 
between  far-apart  hydrophones.  The  region  from  each  nearby  hydrophone  pair  that  had  the 
largest  area  was  used  for  the  correlation.  We  chose  to  use  the  region  with  the  largest  area 
for  matching  because  it  gave  better  results  than  using  the  smallest  region.  The  larger 
regions  tended  to  have  higher  correlations  with  regions  from  the  far-apart  hydrophones. 

Each  pair  of  regions  from  the  nearby  hydrophones  were  matched  to  all  possible  pairs 
of  regions  from  the  far-apart  hydrophones.  The  matchings  were  put  into  a  list  in  decreasing 
order  according  to  their  goodness-of-fit.  A  time-shift  transform  was  then  performed  on  the 
best  three  matches  for  all  of  the  regions. 

We  developed  the  time-shift  transform  from  an  inspiration  given  to  us  by  the  Hough 
transform.  The  Hough  transform  is  a  vision  processing  technique  which  is  typically  used 
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for  finding  lines  and  curves  in  an  image.  One  of  its  best  advantages  is  that  gaps  in  lines  and 
a  noisy  image  have  little  effect  on  the  results  of  the  transform  (Ballard,  1982,  p.  123).  We 
were  not  looking  for  lines,  but  trends  in  the  time  delays  that  were  exhibited  by  the  matches 
of  the  signals.  Finding  trends  in  the  time  delays,  in  an  image  where  there  are  several  very 
close  goodness-of-fit  values  for  time  delays,  is  analogous  to  finding  lines  in  a  noisy  image. 
We  divided  up  the  total  possible  time-delay  for  correlation  of  signals  between  the  far-apart 
hydrophones  into  bins  that  covered  approximately  25  seconds  each.  This  gave  us  20  bins 
for  the  time-delay  that  we  worked  with  on  this  data  set.  We  chose  this  number  because  25 
seconds  was  a  typical  length  for  a  whale  moan.  For  each  region,  the  goodness-of-fit  value 
for  each  of  the  top  three  matching  regions  was  added  to  the  appropriate  time-delay  bin. 
Each  bin  was  then  normalized  to  0  by  subtracting  the  smallest  bin  value  from  all  of  the  bins. 
An  omnidirectional  noise  level  would  show  up  as  the  same  value  in  each  of  the  bins  in  the 
time  shift  array.  The  normalization  to  0  effectively  eliminated  these  noise  values. 

After  normalization  to  0,  each  bin  was  replaced  with  its  square  root.  The  square  root 
was  applied  because  each  bin  is  made  up  of  the  goodness-of-fit  values  from  both  sets  of 
hydrophones.  A  large  set  of  strongly  correlated  signals  would  overshadow  a  smaller  set  of 
correlated  signals  because  of  the  contributions  of  both  sets  of  hydrophones  to  the  time-shift 
array.  If  each  bin  were  divided  by  two,  there  would  be  no  net  overall  effect,  but  by  taking 
the  square  root,  a  dampening  effect  is  applied  to  the  strongly  correlated  values. 

After  each  bin  was  replaced  by  its  square  root,  each  bin  was  normalized  to  1  by 
dividing  it  by  the  maximum  bin  value.  The  result  was  a  time-shift  array  with  values  in  the 
range  from  0  to  1.  This  time-shift  array  represented  the  relative  frequencies  of  time  delays 
for  the  correlations. 

Once  the  time-shift  array  was  determined,  the  goodness-of-fit  value  for  each  match 
between  the  far-apart  hydrophones  was  multiplied  by  the  appropriate  value  from  the  time- 
shift  array.  This  application  of  the  time-shift  array  to  the  best-fit  values  decreased  the 
likelihood  that  a  goodness-of-fit  value,  for  a  time  delay  that  had  little  evidence  to  support 
it,  would  dominate  a  similar  goodness-of-fit  value,  for  a  time  delay  that  had  a  lot  of 
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evidence  to  support  it.  All  far-apart  matches  were  then  rearranged  according  to  the  new  set 
of  goodness-of-fit  values. 

After  the  application  of  the  time-shift  array,  each  pair  of  regions  from  one  pair  of 
nearby  hydrophones  had  its  best  fit  matched  against  the  best-fit  regions  from  the  far-apart 
pair  of  hydrophones.  The  best  fit  of  a  pair  of  nearby  hydrophones  was  a  pair  of  nearby 
regions  from  the  far-apart  hydrophones.  If  the  best  fit  of  the  far-apart  pair  of  regions  was 
the  original  set  of  regions,  then  those  two  sets  of  regions  were  classified  as  being  correlated. 
After  finding  the  best  fits,  each  of  the  matched  regions  was  removed  from  the  list  of 
possible  matches  and  a  second  set  of  best  fits  was  correlated.  Two  passes  of  the  data  set 
were  sufficient  to  correlate  nearly  all  of  the  far-apart  regions. 
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V.  IDENTIFICATION  PROGRAM 

A.  INTRODUCTION 

The  identification  program  identifies  whale  moans,  t-phase  signals,  and  calibration 
signals.  This  chapter  describes  the  identification  program  in  detail,  including  the  program 
input  and  output,  program  structure,  and  detailed  descriptions  of  each  program  component. 

B.  GENERAL  DESCRIPTION 

The  identification  program  was  written  in  Prolog.  Prolog  is  a  language  that  is 
extensively  used  in  artificial-intelligence  applications.  Some  attractive  features  of  Prolog 
are  its  logic-like  syntax,  its  ability  to  backtrack  through  previous  steps,  and  its 
multidirectional  reasoning  (Rowe,  1988,  p.  xiv).  It  was  chosen  as  the  language  for  the  first 
part  of  this  thesis  primarily  for  these  reasons.  It  was  also  chosen  because  Professor  Rowe 
had  a  library  of  computer-vision  software,  written  in  Prolog,  that  could  be  used  as  a 
foundation  for  the  thesis  research.  One  of  the  characteristics  of  Prolog  programs,  including 
the  program  written  for  this  thesis,  is  that  there  is  no  main  program  as  in  a  traditional 
programming  language.  The  program  is  made  up  of  many  short  programs,  usually  just  a 
few  lines  long. 

The  identification  program,  diagramed  in  Figure  6,  is  approximately  1 150  lines  long, 
including  the  Prolog  code  and  comments.  I  wrote  approximately  750  lines  and  Professor 
Rowe  wrote  the  other  450  lines.  There  is  an  additional  library  of  vision  processing 
programs  that  were  written  entirely  by  Professor  Rowe.  Some  parts  of  this  library  are  called 
on  extensively  by  the  programs  in  this  thesis.  This  library  contains  approximately  750  lines 
of  Prolog  code. 

The  identification  program  (see  Appendix  C)  starts  with  a  listing  of  the  Fast  Fourier 
Transform  (FFT)  magnitudes  which  have  been  computed  from  the  raw  time-voltage  data 
using  a  C  program.  The  FFT  magnitudes  are  in  a  time-frequency  order.  Each  line  contains 
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the  magnitudes  for  one  second  of  time  and  each  of  the  64  magnitudes  on  a  line  correspond 
to  one  Hertz  of  frequency  in  ascending  order. 

The  primary  purpose  of  the  identification  program  is  to  take  the  FFT  magnitudes, 
create  a  digital  black-and-white  picture  from  them,  and  identify  the  different  shapes  that 
appear  in  the  picture.  The  program  goes  through  two  phases  of  computer-vision  processing 
in  order  to  achieve  a  result.  The  low-level  processing  works  with  the  pixels  of  the  entire 
picture  to  develop  regions.  The  high-level  processing  performs  feature  extraction  using  the 
individual  regions  and  classifies  regions  and  groups  of  regions. 

There  are  two  sets  of  output  from  the  identification  program.  The  first  set  of  output  is 
a  file  which  contains  a  list  of  the  grouped  regions,  the  parameters  of  these  groups,  and 
horizontal  and  vertical  lines  that  have  been  found  in  the  picture.  The  first  set  of  output  also 
contains  a  listing  of  the  classifications  of  all  of  the  groups  of  regions.  These  classifications 
are:  whale,  near-field  t-phase,  far-field  t-phase,  and  unknown.  The  parameters  of  the  groups 
are:  minimum  and  maximum  frequency,  start  and  stop  time,  area,  density,  and  average 
intensity.  These  parameters  were  chosen  because  they  provide  a  good  overall  description 
of  the  region.  The  average  intensity  is  defined  as  the  sum  of  all  of  the  magnitudes  of  the 
Fourier  transform  for  each  pixel  in  the  group  divided  by  the  area  of  the  group.  The  density 
is  the  pixel  density  in  the  bounding  box  delineated  by  the  minimums  and  maximums  of 
frequency  and  time.  In  addition  to  the  previous  items,  the  first  set  of  output  also  contains 
an  orientation  for  each  unknown  group  of  regions.  A  straight  line  is  fit  to  the  pixels  in  each 
unknown  group  in  a  least-squares  manner  and  the  orientation  of  that  line  is  listed. 

The  second  set  of  output  is  a  list  of  pixels  from  each  of  the  identified  groups.  This 
second  set  of  output  is  written  in  such  a  manner  as  to  be  easily  readable  by  the  correlation 
program.  Each  group  of  regions  is  a  list  enclosed  in  parentheses  with  a  letter  code  indicating 
the  identification  of  the  region  as  the  first  element  of  that  list. 
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C.    BLOCK  DIAGRAMS 
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Figure  6:  Top  Level  Diagram  of  Identification  Program 
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Figure  7:  Diagram  of  Low-Level  Vision  Processing. 
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Figure  8:  Diagram  of  High-Level  Vision  Processing. 
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D.    LOW-LEVEL  VISION  PROCESSING  COMPONENTS 

The  low-level  processing  is  the  first  major  section  of  the  identification  program.  It 
consists  of  10  steps  which  will  take  as  input  the  time-frequency  data  and  turn  it  into  a  set 
of  identified  cohesive  regions. 

When  a  Fast  Fourier  Transform  (FFT)  is  run  on  the  original  time-amplitude  data,  it 
produces  a  set  of  data  that  is  three  dimensional  in  nature.  For  each  time  and  frequency  point, 
there  is  a  number  that  represents  the  strength  of  the  signal  at  that  point.  For  the  data  set  that 
was  provided  for  this  project,  the  original  data  was  collected  at  128  Hz.  FFTs  were  run  on 
every  128  points  to  get  a  1  second  time  resolution  and  a  1  Hz  frequency  resolution.  The  FFT 
returns  a  frequency  array  of  128  points,  but  only  64  points  are  saved  due  to  the  symmetry 
of  the  spectrum  (Press,  1988,  pg.  403). 

The  first  step  in  the  low-level  processing  takes  the  time-frequency  data  and  develops 
a  binary  map  of  pixel  data  that  is  based  on  a  moving  threshold  value.  The  objective  of  the 
first  step  is  to  develop  a  black  and  white  image  that  has  enough  pixels  turned  on  so  that 
shapes  can  be  determined,  but  not  so  many  pixels  turned  on  so  that  multiple  shapes  become 
merged.  Frequency  values  that  were  above  the  threshold  level  caused  the  corresponding 
pixel  to  be  turned  on  and  frequency  values  that  were  below  the  threshold  level  caused  the 
corresponding  pixel  to  be  turned  off.  We  experimentally  determined  that  the  threshold 
should  be  adjusted  to  keep  the  number  of  turned-on  pixels  between  15%  and  25%  of  the 
number  of  pixels  in  the  frequency  range  for  one  second.  These  values  allowed  the  following 
steps  to  create  a  set  of  cohesive  identifiable  regions.  Upper  and  lower  limits  were  set  on  the 
moving  threshold  value  so  that  extremely  noisy  periods  and  extremely  quiet  periods  would 
be  represented.  Figure  9(a)  shows  some  whale  moans  that  are  the  result  of  the  first  step.  The 
programs  for  creating  a  binary  map  of  pixel  data  based  on  a  fixed  threshold  were  written 
by  Professor  Rowe  and  modified  by  the  author  for  the  moving  threshold  value. 

The  second  step  in  the  low-level  processing  takes  the  black  and  white  pixel  image  that 
is  the  result  of  step  one  and  turns  off  all  pixels  that  are  turned  on  and  are  not  strongly 
connected.  This  step  removes  much  of  the  spurious  noise  in  the  image.  Figure  9(b)  shows 
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the  result  of  the  second  step.  The  program  for  performing  the  second  step  was  written  by 
Professor  Rowe.  The  third  step  in  the  low-level  processing  performs  region  growing  by 
turning  on  pixels  in  order  to  fill  in  gaps  across  rows,  columns,  and  diagonals.  Figure  9(c) 
shows  the  result  of  the  third  step.  The  effect  of  this  region  growing  step  is  to  make  fuller 
shapes. 

The  fourth  step  and  sixth  step  are  the  same,  as  are  the  fifth  and  seventh  steps  in  the  low- 
level  processing.  In  the  fourth  and  sixth  steps,  pixels  which  are  probably  noise  or  are  man- 
made  are  marked  with  special  symbols  so  that  they  can  be  removed  in  the  next  step. 
Presumed  man-made  pixels  are  those  turned-on  pixels  that  create  a  narrow  frequency  band 
for  2  or  more  seconds.  Also  fitting  this  category  is  the  calibration  signal,  which  has  a 
distinctive  set  of  frequencies.  Pixels  that  are  considered  to  be  noise  are  those  turned-on 
pixels  that  are  not  strongly  connected  in  frequency,  regardless  as  to  whether  they  are 
strongly  connected  in  time.  The  fifth  and  seventh  steps  in  the  low  level  processing  simply 
remove  all  of  the  specially  marked  symbols  from  the  previous  steps.  The  result  of  these  four 
steps  is  a  black  and  white  picture  that  has  its  major  features  sharpened  and  most  of  the  noise 
surrounding  these  features  removed.  Figure  10  shows  the  results  of  these  steps.  The 
diagonal  lines  with  a  positive  slope  indicate  pixels  that  were  identified  as  man-made  and 
the  diagonal  lines  with  a  negative  slope  indicate  pixels  that  were  identified  as  calibration 
signals. 

The  eighth  step  in  the  low-level  processing  performs  region  clumping  by  taking  the 
black  and  white  picture  and  associating  a  region  number  with  each  pixel.  All  pixels  that  are 
either  weakly  connected  or  strongly  connected  are  associated  with  the  same  region.  The 
result  of  this  clumping  is  a  picture  that  has  numbers  in  the  place  of  all  of  the  previously 
turned  on  pixels.  The  programs  for  performing  the  eighth  step  were  written  by  Professor 
Rowe. 

The  ninth  step  in  the  low-level  processing  removes  the  concept  of  a  picture  that  has 
pixels  that  are  turned  on  or  off  and  replaces  this  with  the  concept  of  a  set  of  regions  that 
have  time  and  frequency  values  associated  with  them.  The  programs  for  performing  the 
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ninth  step  were  written  by  Professor  Rowe.  The  tenth,  and  final,  step  in  the  low-level 
processing  goes  back  to  the  original  data  file  and  associates  the  strength  of  the  signal  at  each 
turned-on  pixel  with  the  time-frequency  values  for  each  region. 

There  is  no  longer  an  image  for  the  vision  processing  methods  to  work  with.  Instead, 
there  is  a  set  of  well  defined  regions  that  have  time,  frequency,  and  strength  values 
associated  with  them.  In  low-level  vision  processing,  each  element  of  a  picture  is  processed 
in  the  same  way  (Charniak,  1985,  pg.  95).  Even  though  some  of  the  follow-on  processing 
will  process  each  element  of  each  region  in  the  same  way,  there  aren't  any  tumed-off  pixels 
to  process,  so  all  of  the  following  processing  is  considered  to  be  high-level  vision 
processing. 
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Figure  9:  Results  of  the  First  Three  Steps  of  Low-Level  Processing. 
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Figure  10:  Results  of  Low-Level  Processing  Steps  Four  To  Seven 
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E.    HIGH-LEVEL  VISION  PROCESSING  COMPONENTS 

The  first  step  in  the  high-level  processing  finds  the  area,  circumference,  length,  and 
width  of  each  region.  The  area  of  a  region  is  the  total  number  of  pixels  in  that  region.  The 
circumference  of  a  region  is  the  number  of  pixels  in  that  region  that  are  not  strongly 
connected  to  four  other  pixels.  The  length  of  a  region  corresponds  to  the  total  amount  of 
time  that  a  sound  occurs,  and  the  width  of  a  region  corresponds  to  the  total  bandwidth  of  a 
signal. 

The  second  step  in  the  high-level  processing  is  a  computational  convenience.  It  was 
found  that  by  specially  noting  the  circumference  pixels,  that  the  process  of  grouping 
regions  was  much  faster.  This  step  finds  each  pixel  in  each  region  that  is  a  circumference 
pixel  and  asserts  it  as  an  outer  pixel. 

The  third  step  finds  nearby  regions  and  groups  them.  If  the  square  of  the  distance 
between  any  two  outer  pixels  of  two  regions  is  less  than  13,  then  the  two  regions  were 
placed  in  the  same  group.  The  distance  between  pixels  is  related  to  their  time-frequency 
value.  The  number  13  was  derived  experimentally. 

The  fourth  step  characterizes  the  groups.  For  each  group,  the  start  and  stop  times  are 
determined  and  the  minimum  and  maximum  frequencies  are  determined.  Each  group  also 
has  its  total  area  (total  number  of  pixels)  determined  along  with  its  density  and  average 
intensity.  Density  was  defined  as  total  area,  which  is  determined  by  the  bounding  box, 
divided  by  the  group  area,  which  is  a  count  of  the  number  of  pixels  in  the  group.  The 
average  intensity  was  determined  from  the  intensity  values  that  were  associated  with  each 
pixel  in  the  last  step  of  the  lower-level  processing. 

The  fifth  step  of  the  high-level  processing  finds  horizontal  lines.  Horizontal  lines  are  a 
collection  of  pixels  that  are  outer-pixels  of  regions  and  extend  over  a  significant  period  of 
time  without  changing  in  frequency.  These  lines  could  indicate  the  presence  of  a  man-made 
signal  that  was  not  removed  in  the  low-level  processing.  The  sixth  step  is  similar  to  the  fifth 
step  except  that  it  finds  vertical  lines.  These  are  lines  that  cover  many  frequencies  at  one 
time.  The  vertical  lines  could  also  be  an  indicator  for  a  man-made  signal  such  as  an 
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underwater  explosion.  The  seventh  step  in  the  high-level  processing  finds  vertices  from  the 
intersections  of  the  vertical  and  horizontal  lines.  These  three  steps  are  most  often  performed 
as  low-level  vision  processing  functions  (Ballard,  1982,  pg.  119),  but  we  felt  that  it  was 
more  appropriate  to  perform  these  steps  in  the  high-level  processing  so  that  region 
identifications  could  be  associated  with  these  features. 

The  eighth  step  is  a  first  pass  at  finding  whale  signals.  The  criteria  for  a  group  to  be 
called  a  whale  on  the  first  pass  is  that  it  has:  a  minimum  frequency  greater  than  20  Hz,  a 
time  span  between  12  and  60  seconds,  an  area  between  50  and  500  pixels,  and  a  density 
between  8  and  75%. 

The  ninth  step  in  the  high-level  processing  is  finding  t-phase  signals.  First,  all  potential 
t-phases  are  found  with  the  simple  criteria  that  they  have  an  area  greater  than  5  pixels  and 
a  minimum  frequency  that  is  less  than  10  Hz.  T-phases  are  then  divided  into  far-field  and 
near-field  earthquakes.  Far-field  earthquakes  are  considered  to  be  those  that  have  a 
maximum  frequency  of  less  than  25  Hz.  See  Chapter  II  for  a  discussion  of  frequency 
attenuation.  Far-field  earthquakes  also  have  the  tendency  to  have  many  disconnected 
groups  of  regions.  If  any  two  groups  of  regions  are  determined  to  be  far-field  earthquakes 
and  they  are  separated  by  a  time  period  of  less  than  9  seconds,  then  they  are  determined  to 
be  a  part  of  the  same  earthquake.  The  near- field  earthquakes  are  all  earthquakes  that  are  not 
far-field  earthquakes. 

The  tenth  step  in  the  high-level  processing  is  to  place  all  of  the  remaining  groups  of 
regions  that  have  an  area  greater  than  seven  into  a  category  called  "unknown  groups".  For 
each  of  these  unknown  groups,  further  determinations  are  made  about  their  shapes.  A 
straight  line  is  least-squares  fit  through  the  pixels  of  the  group  and  the  angle  of  that  line  in 
the  frequency-time  plane  is  determined.  With  this  line  established,  the  mean  distance  of  the 
pixels  from  the  line  is  determined,  along  with  the  variance. 

The  eleventh  step  in  the  high-level  processing  is  to  check  each  unknown  group  to  see 
if  it  might  be  part  of  a  whale  moan.  The  previous  processing  was  not  including  many  of  the 
upper  frequency  sounds  of  the  whale  moans  because  the  regions  of  the  higher  frequency 
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sounds  were  separated  by  too  many  turned-off  pixels  from  the  regions  making  up  the 
whale-moan  group.  If  the  maximum  frequency  of  a  whale  moan  is  within  8  Hertz  of  the 
minimum  frequency  of  an  unknown  group  and  if  the  ending  time  of  a  whale  moan  is  within 
14  seconds  of  the  starting  time  of  an  unknown  group,  then  the  unknown  group  is  removed 
from  the  list  of  unknown  groups  and  a  combined  whale  group  is  created. 

The  twelfth  and  final  step  in  the  high-level  vision  processing  is  writing  the  results  to 
files.  An  explanation  of  the  output  is  given  in  the  general  program  description  section  of 
this  chapter. 

F.    ABANDONED  ALGORITHMS 

The  whale  moans  could  be  classified  using  other  types  of  parameters.  Another 
parameter  that  can  help  to  define  a  whale  moan  is  the  typical  angle  that  a  line  would  make 
if  it  were  passed  through  the  signal  using  a  least  squares  fit.  This  would  describe  the 
tendency  of  the  whale  moan  to  upsweep  in  time  through  a  frequency  band  of  about  40 
Hertz.  This  line-fitting  approach  was  tried  at  one  point  in  the  investigation,  but  it  was  found 
to  be  too  time  consuming. 

The  calibration  signal  was  originally  left  for  high-level  processing.  It  was  found  that 
frequent  small  gaps  in  the  signal  and  variations  in  the  upper  frequency  ranges  of  the  signal, 
combined  with  abutting  regions  from  other  sources  at  the  beginning  and  end  of  the 
calibration  signal,  caused  it  to  be  very  difficult  to  identify  during  the  high  level  processing, 
but  the  low-level  processing  identifies  the  calibration  signal  very  easily  on  a  line  by  line 
basis. 
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VI.  CORRELATION  PROGRAM 

A.  INTRODUCTION 

The  correlation  program  has  two  goals.  The  first  is  to  correlate  signals  between  nearby 
hydrophones.  The  second  is  to  correlate  the  nearby-correlated  signals  to  far-apart 
hydrophones.  Different  techniques  were  used  for  each  type  of  correlation.  This  chapter 
describes  the  correlation  program  in  detail,  including  the  program  input  and  output, 
program  structure,  data  structures,  and  detailed  descriptions  of  each  program  component. 

B.  GENERAL  DESCRIPTION 

The  correlation  program  (see  Appendix  D)  was  written  in  the  Common  Lisp  Object 
System  (CLOS).  CLOS  is  an  object-oriented  programming  language  which  is  built  on  top 
of  Common  Lisp.  It  is  used  extensively  in  artificial-intelligence  applications  (Koschmann, 
1990,  pg.  5).  We  chose  CLOS  for  this  part  of  the  thesis  because  of  its  ability  to  handle 
complicated  data  structures  and  because  of  the  graphics  capabilities  that  came  with  the 
implementation  that  was  supplied  with  our  computer  systems.  Our  implementation  was 
Allegro  Common  Lisp  from  Franz. 

There  are  approximately  1000  lines  of  code  in  the  program  divided  up  into  five  files 
that  are  separated  by  function.  Comments  are  sparse  in  the  code  because  descriptive  names 
were  used  for  the  variables,  making  comments  redundant.  It  is  possible  to  have  global 
variables  in  a  CLOS  program,  but  it  is  not  considered  good  programming  practice,  so  there 
are  none  in  this  program. 

The  correlation  program  (see  Appendix  D)  starts  by  creating  and  initializing  instances 
of  a  blackboard  data  structure  for  each  pair  of  nearby  hydrophones.  Then  it  creates  and 
initializes  a  data  structure  for  the  graphic  output.  The  blackboard  control  program  is  then 
called  sequentially  for  each  nearby  pair  of  hydrophones.  After  each  set  of  nearby 
hydrophones  is  processed,  regions  are  correlated  between  the  far-apart  hydrophones. 
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C.    INPUT  AND  OUTPUT 

The  input  to  the  program  consists  of  lists  of  regions  with  the  identification  of  the  region 
type  as  the  first  element  of  the  list.  This  is  the  output  from  the  identification  program.  There 
are  four  hydrophones  to  which  sounds  are  correlated.  The  identification  program  only 
processes  the  sounds  from  one  hydrophone,  so  it  must  be  run  four  times  covering  the  same 
time  period  during  each  run,  in  order  to  produce  the  files  that  are  used  by  the  correlation 
program. 

The  output  of  the  correlation  program  is  an  X-windows  plot  of  identified  and 
correlated  regions  in  a  time-frequency  diagram  (Figure  1 1(a)  and  (b)).  The  vertical  axis  is 
frequency  from  0  to  64  Hertz.  There  are  four  hydrophones  shown  in  the  plot,  so  there  are 
four  vertical  axes,  all  with  the  same  scale.  The  nearby  pairs  of  hydrophones  are  shown 
adjacent  to  each  other  on  the  plot,  with  the  line  between  each  pair  corresponding  to  64  Hertz 
on  the  lower  hydrophone  plot  and  0  Hertz  on  the  upper  hydrophone  plot. 

The  horizontal  axis  is  time.  There  is  approximately  20  minutes  of  sounds  in  each  plot. 
Each  minute  from  the  start  of  the  plot  is  denoted  by  a  tic  mark  below  the  lower  hydrophone 
in  each  pair.  All  hydrophones  were  processed  over  the  same  time  period  so  there  are  no  time 
offsets  of  the  regions  between  hydrophones. 

Regions  are  identified  by  a  letter  slightly  above  and  on  the  right  side  of  the  region. 
Whales  are  identified  with  a  "W",  t-phases  with  a  "Q",  unknowns  with  a  "U",  and  others 
with  an  "O".  Correlated  regions  have  a  line  drawn  between  them.  The  endpoints  of  the  lines 
were  determined  by  the  middle  pixel  in  the  list  of  pixels  for  the  region,  so  the  angle  of  the 
line,  especially  between  the  nearby  hydrophones  does  not  represent  the  exact  time  offset 
for  the  correlation. 

The  numbers  above  the  upper  pair  of  hydrophones  and  below  the  lower  pair  of 
hydrophones  are  the  time  offsets  for  the  near-field  correlations.  They  are  in  whole  seconds 
and  are  referenced  to  the  left  side  of  the  graph. 
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Figure  11:  Graphs  Produced  by  the  Correlation  Program. 
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Figure  12:  Diagram  of  Control  Program. 
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Figure  13:  Diagram  of  Blackboard  Controller. 
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Figure  14:  Diagram  of  Process  New  Region  Program. 
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Figure  15:  Diagram  of  Match  Far  Regions  Program. 
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E.    DATA  STRUCTURES 

There  are  four  classes  of  objects  in  the  correlation  program.  They  are:  the  blackboard 
class,  the  hydrophone  class,  the  region  class,  and  the  screen  class.  Each  time  the  program 
is  run,  there  are  two  blackboard  objects,  two  hydrophone  objects  for  each  blackboard 
object,  many  region  objects,  and  one  screen  object. 

The  slots  in  the  blackboard  class  are:  phone  1,  phone2,  screen-position,  allowable- 
time-difference,  unmatched-regions,  definite-unmatched-regions,  possible-matched- 
regions,  definite-matched-regions,  possible-region-matches,  definite-region-matches, 
possible-region-match-time,  and  last-definite-matched-region-time.  The  blackboard  class 
is  a  superclass  of  the  hydrophone  class  through  the  phone  1  and  phone2  slots. 

The  screen  position  slot  in  the  blackboard  class  is  initialized  with  the  blackboard  and 
indicates  whether  the  hydrophones  are  to  be  plotted  on  the  upper  part  of  the  screen  or  the 
lower  part  of  the  screen.  It  remains  constant  for  the  life  of  the  object.  This  slot  is  necessary 
because  there  are  two  blackboard  objects  in  the  program.  The  allowable  time  difference  slot 
is  also  initialized  with  the  blackboard  and  remains  constant  for  the  life  of  the  object.  It  is 
determined  from  input  parameters  that  state  the  relative  positioning  of  the  hydrophones,  the 
assumed  speed  of  sound  in  the  deep  sound  channel,  and  an  error  factor  due  to  uncertainty 
about  the  deep  sound  channel  characteristics. 

The  unmatched  regions  slot  in  the  blackboard  class  is  a  list  of  the  region  identifications 
of  the  regions  that  have  not  been  matched  to  other  regions  and  still  fit  within  a  time  window 
where  they  might  possibly  be  matched.  The  definite-unmatched-regions  slot  is  a  list  of  the 
region  identifications  of  the  regions  that  have  not  been  matched  to  other  regions  and  no 
longer  fit  within  a  time- window  where  they  might  possibly  be  matched. 

The  possible-matched-regions  slot  in  the  blackboard  class  is  a  list  of  the  region 
identifications  that  have  tentatively  been  matched  to  another  region,  but  still  fit  within  a 
time  window  where  another  region  might  possibly  be  matched  to  them.  The  definitely- 
matched-regions  slot  is  a  list  of  the  region  identifications  that  have  been  uniquely  matched 
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to  another  region  and  no  longer  fit  within  a  time  window  where  another  region  might 
possibly  be  matched  to  them. 

The  possible-region-matches  and  definite-region-matches  slots  in  the  blackboard  class 
are  lists  of  lists  of  the  region-identifications  that  are  matched,  combined  with  the  goodness- 
of-fit  value  and  the  time  delay  between  the  region  match.  The  possible-region-match-time 
and  last-definite-region-match-time  slots  are  used  by  the  blackboard  controller  to 
determine  whether  it  should  implement  the  definite- matched-region  knowledge  source. 

The  hydrophone  class  is  a  sub-class  of  the  blackboard  class.  It  contains  the  following 
slots:  file,  blackboard-id,  phone-id,  latitude,  and  phone-time.  The  file  slot  is  the  name  of  the 
file  that  contains  the  region  data  for  one  hydrophone.  It  is  supplied  at  the  beginning  of  the 
program  and  remains  constant  for  the  life  of  the  object.  The  blackboard-id  slot  corresponds 
to  the  screen-position  slot  from  the  blackboard  class.  The  latitude  slot  is  supplied  as  input 
to  the  initialization  of  the  hydrophone  and  represents  the  relative  position  of  the 
hydrophone  to  the  other  hydrophones.  The  phone-time  slot  corresponds  to  the  start  time  of 
the  most  recently  retrieved  region  from  the  input  file. 

The  region  class  contains  the  following  slots:  blackboard-id,  phone-id,  region-id, 
region-type,  pixels,  time-min,  time-max,  freq-min,  freq-max,  and  area.  The  blackboard-id 
slot  corresponds  to  the  screen-position  slot  in  the  blackboard  class.  The  phone-id  slot  is  set 
to  the  phone-id  of  the  relevant  phone  object.  The  region-id  slot  is  a  unique  machine- 
generated  identification  by  which  the  region  can  be  identified.  The  region-type  slot  is  a 
letter  corresponding  to  the  identified  type  of  region.  The  region  type  is  the  first  element  of 
the  input  list.  The  pixels  slot  is  a  list  of  time  and  frequency  values  for  each  pixel  in  the 
region.  The  pixels  make  up  the  rest  of  the  input  for  a  region  after  the  region  type.  The  time- 
min,  time-max,  freq-min,  and  freq-max  slots  are  the  extremes  of  time  and  frequency  for  the 
region.  They  are  calculated  after  being  read.  The  area  slot  is  the  number  of  pixels  in  the 
region. 

The  screen  class  has  the  following  slots:  id,  borders,  left,  bottom,  width,  height,  title, 
activate-p,  lower-y-text-position,  upper-y-text  position.  The  id  slot  is  the  address  of  the 


42 


plot.  The  computer  sends  plotting  commands  to  this  address.  The  borders,  left,  bottom, 
width,  and  height  slots  in  the  screen  class  are  for  physical  positioning  of  the  plot  on  the 
graphics  screen.  The  two  text-position  slots  are  for  positioning  text  on  the  screen. 

F.    PROGRAM  COMPONENTS 

The  control  program  performs  the  initialization  of  the  data  structures  and  calls  on  the 
major  controlling  components.  Two  blackboard  objects  and  a  screen  object  are  created  and 
initialized.  Each  blackboard  object  initialization  call  includes  the  names  of  the  two  files  that 
contain  the  region  data  from  the  nearby  hydrophones,  the  relative  position  of  these 
hydrophones  in  nautical  miles,  and  the  relative  screen  position  to  which  the  output  will  be 
plotted.  The  relative  physical  positions  were  called  "latitude"  in  order  to  avoid  confusion 
with  the  screen  position  of  the  hydrophone  output.  After  the  blackboard  objects  are 
initialized,  a  screen  object  is  initialized.  The  initialization  of  the  screen  object  causes  the 
graphics  screen  to  display  a  plot.  Once  this  is  done,  the  blackboard  controller  for  the  first 
blackboard  is  invoked.  This  causes  the  first  two  nearby  hydrophones  files  to  be  read  and 
their  regions  to  be  correlated  and  displayed.  After  the  data  set  for  the  first  set  of 
hydrophones  has  been  exhausted,  the  blackboard  controller  for  the  second  blackboard  is 
invoked.  With  two  blackboard  objects  of  nearby  correlated  hydrophones  completed,  the 
control  program  calls  on  the  far-apart-hydrophone  matching  program  to  match  the  matched 
regions  from  the  two  blackboards. 

When  a  blackboard  object  is  created,  the  computer  associates  a  name  with  the  object 
and  sets  aside  memory  for  that  object  to  use.  In  the  initialization  of  the  blackboard,  each  of 
the  two  hydrophones  are  started.  This  involves  opening  their  files  for  reading  and  adding 
the  information  about  the  hydrophone's  latitude  and  screen  position  to  the  hydrophone  data 
structure.  The  blackboard's  screen  position  is  added  to  its  data  structure,  and  the  allowable 
time  difference  is  determined  from  the  latitude  difference  of  each  hydrophone  divided  by 
the  assumed  sound  speed  and  multiplied  by  an  uncertainty  factor. 
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After  the  blackboard  objects  are  initialized,  the  screen  object  is  initialized.  During  the 
initialization  of  the  screen  object,  the  plot  window  is  drawn  to  the  screen,  the  lines 
delineating  the  minimum  and  maximum  frequencies  for  the  hydrophones  are  drawn,  as  are 
tic  marks  for  each  minute  in  time. 

Once  the  initialization  is  done,  the  blackboard  controller  is  called  for  each  blackboard. 
The  blackboard  controller  is  a  recursive  program.  It  first  checks  the  phone-time  slot  for 
each  hydrophone  and  gets  the  next  region  from  the  hydrophone  with  the  oldest  time.  The 
hydrophones  stay  synchronized  in  time  by  getting  regions  in  this  manner.  After  a  region  is 
retrieved  from  the  hydrophone  file,  the  controller  calls  on  a  program  to  process  the  new 
region.  Once  the  process-new-region  program  returns,  the  controller  checks  the  time 
difference  between  the  last  possible  region  match  and  the  last  definite  region  match  against 
the  allowable  time  difference.  If  sufficient  time  has  passed,  the  controller  calls  the  program 
for  moving  region  matches  from  the  possible-matched-regions  slot  to  the  definite-matched- 
regions  slot.  After  this  is  done,  the  blackboard  controller  calls  itself.  This  continues  until 
both  hydrophone  files  have  been  exhausted. 

The  process-new-region  program  starts  by  calling  on  a  program  to  move  regions  from 
the  unmatched-region  slot  to  the  unmatchable-regions  slot.  Regions  are  moved  to  the 
unmatchable-regions  slot  when  they  can  no  longer  be  matched  to  any  new  regions  because 
the  start  times  are  too  far  apart.  The  process-new-region  program  then  attempts  to  match 
the  new  region  to  all  regions  from  the  other  hydrophone  in  the  unmatched-region  slot.  A 
time-shift  and  overlap  technique  is  employed  for  matching  regions.  The  time  element  of 
each  pixel  in  one  region  is  modified  so  that  it  is  time-correlated  to  the  region  to  be  matched, 
then  a  goodness-of-fit  is  determined  from  the  amount  of  overlap  between  the  two  regions. 
If  a  match  is  made,  both  regions  are  moved  to  the  possible-matched-regions  slot.  After  the 
unmatched  regions  have  been  checked,  an  attempt  is  made  to  match  the  new  region  to  all 
regions  in  the  possible-matched  regions  slot. 

After  the  control  program  calls  the  blackboard  controller  for  each  of  the  two  pairs  of 
nearby  hydrophones,  it  calls  the  program  for  matching  far  apart  hydrophones.  The  far- 
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match  program  first  takes  the  list  of  definite-region-matches  from  one  blackboard  and 
matches  each  matched  pair  to  all  possible  regions  from  the  list  of  definite-region- matches 
from  the  other  blackboard.  This  produces  a  list,  for  each  nearby  hydrophone  pair,  of  far- 
apart  region  pairs  with  a  goodness-of-fit  and  time-delay.  The  number  of  possible  regions 
for  matching  is  constrained  by  the  travel  time  of  sound  between  the  two  regions.  After  the 
far-apart  matching  is  done  for  the  regions  from  one  blackboard,  it  is  done  for  the  other.  Now 
there  are  two  far-apart  match  lists,  with  each  matched  region  from  each  pair  of  nearby 
hydrophones  matched  to  all  possible  regions  from  the  other  set  of  nearby  hydrophones  with 
the  goodness-of-fit  and  associated  time  delay.  Each  list  of  possible  matches  for  each 
hydrophone  pair  is  ordered,  from  best  to  worst,  by  the  goodness-of-fit  parameter.  The  time- 
shift  transform  is  then  determined  from  the  matched  lists.  The  time-shift  transform  is  then 
applied  to  both  matched  lists.  This  modifies  the  goodness-of-fit  numbers  for  each  of  the 
region  matches  for  each  far-apart  pair  of  matches.  This  modifies  the  ordering  of  the  list  for 
each  hydrophone,  so  each  hydrophone  is  re-ordered,  from  best  to  worst,  by  the  goodness- 
of-fit  parameter.  Once  this  is  done,  a  get-best-matches  program  is  called  to  determine  the 
best  matches  between  the  two  sets  of  far-apart  hydrophones.  It  does  this  by  finding  all  of 
the  one-to-one  correspondences  between  the  best  matches  for  each  pair  of  regions.  Lines 
are  then  drawn  between  the  far-apart  regions  that  have  best  matches.  Once  this  is  done,  all 
of  these  regions  are  removed  from  the  far-apart  match  lists.  Then  the  get-best-matches  is 
called  again,  followed  by  drawing  lines  between  the  additional  best  matches. 

G.    ABANDONED  ALGORITHMS 

The  region  matching  algorithm  for  the  nearby  regions  originally  attempted  to  limit  the 
matchable  time  delay  based  on  the  assumed  speed-of- sound  through  the  water  and  the 
distance  between  the  hydrophones.  This  did  not  work  because  of  the  distortions  that  regions 
could  go  through  during  the  identification  phase.  Some  regions  would  have  their  front  ends 
removed  because  of  the  pixel  trimming  that  goes  on  in  the  low-level  vision  processing.  The 
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fix  for  this  simply  involved  putting  in  artificial  positions  for  the  nearby  hydrophones  that 
were  farther  apart  than  was  actually  the  case. 

When  attempting  to  match  regions  between  the  far  apart  hydrophones,  the  smallest 
region  from  the  matched  pairs  of  nearby  regions  was  used  for  the  time-shift  and  overlap 
comparison.  This  gave  less  satisfactory  results  than  using  the  largest  region  from  the 
matched  pairs. 

Knowledge  sources  in  a  blackboard  architecture  are  usually  independent  and  can 
frequently  act  in  parallel.  We  attempted  to  implement  the  blackboard  controller  to  take 
advantage  of  this  independence  by  having  the  knowledge  sources  called  up  as  separate 
processes.  This  would  probably  have  worked  if  a  multi-processing  computer  was  available 
or  if  multiple  computers  were  used.  Unfortunately,  the  process  controller  grabbed  most  of 
the  processing  time,  giving  very  little  to  the  multiple  processes  that  were  running.  This  was 
mostly  due  to  the  fact  that  every  time  a  pixel  was  drawn  to  the  screen,  an  interrupt  occurred, 
causing  a  context  switch. 
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VII.  DISCUSSION  OF  RESULTS 

A.  INTRODUCTION 

The  identification  program  was  very  successful  in  identifying  t-phase  signals.  Most 
calibration  signals  and  whale  moans  were  also  identified.  There  were  a  few  false 
identification  problems  with  whale  moans  and  also  some  whale  moans  that  were  classified 
as  unidentified  signals. 

The  correlation  program  was  successful  in  correlating  regions  to  nearby  hydrophones. 
It  also  successfully  correlated  regions  to  far  apart  hydrophones  when  all  possible  regions 
had  been  correlated  to  nearby  hydrophones.  False  correlations  tended  to  occur  when  the 
proper  region  on  one  hydrophone  set  was  not  available  for  correlation. 

B.  IDENTIFICATION  PROGRAM 

The  identification  program  was  run  on  a  Sun  Sparcstation  10.  Although  over  10  hours 
of  data  for  16  hydrophones  were  provided  for  this  project,  only  2.2  hours  of  data  for  four 
hydrophones  were  used  due  to  disk  space  limitations.  The  data  set  for  each  run  included 
approximately  1 120  seconds  of  Fourier  transformed  data  from  one  hydrophone.  This  size 
data  set  was  chosen  so  that  full  use  could  be  made  of  the  screen  width  on  the  workstation 
when  displaying  the  results  of  the  correlation  program.  The  average  running  time  was  33.8 
minutes  with  a  sample  standard  deviation  of  13.7  minutes.  The  deviation  is  large  because 
extremely  noisy  time  periods  caused  the  execution  time  to  double  over  time  periods  that 
just  contained  whale  moans.  Execution  time  for  a  set  which  consisted  exclusively  of  whale 
moans  was  approximately  21.7  minutes.  The  average  memory  use  including  program  and 
data  storage  was  14  megabytes  with  a  sample  standard  deviation  of  2.4  megabytes. 
Memory  usage  for  a  set  which  consisted  exclusively  of  whale  moans  was  approximately 
12.2  megabytes. 

A  comparison  was  made  of  the  whale  moans  that  were  visually  identifiable  by  a  human 
and  the  whale  moans  that  were  identified  by  the  program.  There  was  one  data  set  of  1 120 
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seconds  of  sound  that  was  strictly  whale  moans.  During  that  data  set,  the  program  identified 
91.7%  of  156  human-identified  whale  moans  that  were  received  on  four  hydrophones. 
During  periods  of  seismic  activity  and  high  noise,  the  number  of  identifiable  moans  was 
reduced  for  both  the  human  observer  and  for  the  program.  Over  four  data  sets  that  showed 
both  seismic  activity  and  high  noise  covering  a  total  of  4480  seconds,  there  were  469 
human-identified  whale  moans,  68.9%  of  which  were  identified  by  the  program. 

The  number  of  false  identifications  was  negligible.  There  were  two  instances  where  a 
region  was  identified  as  a  whale  moan  when  it  was  actually  two  moans  that  slightly 
overlapped.  There  was  also  a  problem  of  whale  moans  being  mistakenly  identified  in  the 
section  of  the  hydrophone  recording  that  was  extremely  noisy.  Over  a  15  minute  period  of 
noisy  data,  where  nothing  is  identifiable  by  human  vision  on  the  frequency  plots  of  two 
hydrophones,  there  were  30  whale  moans  identified.  The  few  calibration  signals  that 
escaped  elimination  in  the  low-level  processing  phase  of  the  identification  program  were 
also  sometimes  identified  as  whale  moans. 

There  were  many  fewer  t-phase  signals  in  the  data  than  whale  moans,  although  t-phase 
signals  tended  to  have  a  larger  area.  A  single  t-phase  signal  often  consisted  of  many 
regions.  Also,  a  t-phase  signal  can  last  for  several  minutes.  One  signal  that  was  analyzed 
by  the  program  lasted  for  15  minutes  across  two  data  sets.  All  t-phase  signals  were  properly 
identified  by  the  program.  There  were  a  limited  number  of  unidentified  regions  in  the  data 
that  might  have  been  associated  with  a  t-phase  signal,  but  in  every  case,  there  were  several 
small  regions  near  these  unidentified  regions  that  were  identified  as  t-phase  signals.  Also, 
the  author  could  not  be  certain  that  these  regions  were  actually  a  part  of  the  t-phase  signal. 

Most  of  the  calibration  signals  were  removed  in  the  low-level  vision  processing  phase 
of  the  program.  There  was  a  small  number  of  calibration  signals  that  made  it  to  the  high- 
level  processing  and  these  were  identified  as  unknown  signals  or  whale  moans. 

Signals  that  were  not  classified  as  whale  moans,  calibration  signals,  or  t-phases,  were 
classified  as  unknown.  The  identifiable  whale  moans  that  were  not  classified  as  such  were 
all  classified  as  unknown.  Also,  unknown  signals  were  often  small  parts  of  whale  moans 
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that  didn't  get  picked  up  in  the  same  group  as  the  rest  of  the  signal.  During  the  high-noise 
section  of  the  hydrophone  recording,  there  were  many  small  regions  of  noise  that  were 
classified  as  unknown. 

C.    CORRELATION  PROGRAM 

The  correlation  program  was  run  on  a  Sun  Sparcstation  10.  Regions  covering 
approximately  1 120  seconds  of  hydrophone  data  for  each  of  four  hydrophones  were  input 
on  each  run.  The  average  CPU  time  for  the  correlation  program  was  approximately  28 
minutes. 

Correlations  of  signals  between  near-field  hydrophones  are  easily  identifiable  by 
human  inspection.  In  the  first  data  segment,  which  consists  entirely  of  whale  moans  (see 
Figure  1 1),  there  should  be  86  near-field  matches.  The  program  determined  81  (94.1%)  of 
those  matches.  Similar  results  were  obtained  for  all  other  data  segments  where  there  were 
whale  moans  to  be  correlated. 

The  results  were  less  satisfactory  for  t-phase  signals.  The  identification  program 
tended  to  break  t-phase  signals  up  into  several  different  regions.  Often,  the  manner  in  which 
these  regions  were  determined  was  different  for  the  near-field  hydrophones.  Over  the  span 
of  seven  data  sets,  covering  2  hours  and  10  minutes,  there  were  365  regions  identified  as  t- 
phase  signals.  Near-field  correlations  of  these  t-phase  signals  dropped  to  42.7%.  Part  of  the 
reason  for  this  was  that  most  of  the  programming  effort  concentrated  on  the  correlation  of 
whale  moans. 

The  symbolic  correlation  of  signals  to  the  far-field  hydrophones  is  not  obvious  to  the 
human  observer.  When  looking  at  a  plot  of  the  signals  without  the  far-field  correlations 
connected,  one  can  only  visually  correlate  one  or  two  whale  moans  per  data  set.  After  the 
far-field  correlations  are  drawn,  it  is  easier  to  see  that  the  correlations  drawn  by  the  program 
are  probably  correct.  This  human  verification  after-the-fact  is  not  good  enough  to  claim  that 
the  far-field  correlations  are  correct.  Another  approach  was  used  to  verify  the  program 
output. 
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For  each  pair  of  near-field  correlated  whale  moans  in  the  first  data  set,  a  Fast  Fourier 
Transform  correlation  covering  a  16  second  time  period  was  run  against  time  delays  for  the 
possible  matches  to  the  far-field  hydrophones.  Figure  16  shows  a  correlation  of  a  whale 
moan  between  two  hydrophones  with  a  sample  data  set  covering  16  seconds.  The 
correlation  is  between  far- apart  hydrophones  in  Figure  1 1(a)  at  the  indicated  times.  Phone 
1  is  the  lower  hydrophone  in  the  lower  set  of  hydrophones  and  Phone  3  is  the  lower 
hydrophone  in  the  upper  set  of  hydrophones.  The  vertical  axis  is  the  magnitude  of  the 
correlation.  The  horizontal  axis  is  in  units  of  1/128  seconds.  The  correlation  is  for  eight 
seconds  on  either  side  of  the  of  the  selected  time  offsets.  The  correlation  at  the  exact  time 
offset  is  at  the  1024  value  point  on  the  horizontal  axis.  The  maximum  correlation  occurs 
slightly  less  than  a  second  prior  to  that  point  and  has  a  magnitude  of  approximately  6000. 
The  two  time  offsets  in  this  graph  were  selected  because  they  match  time  offsets  for  a 
correlation  chosen  by  the  shape  correlation  program. 


Correlation  of  Phone  1  at  543  Seconds  and  Phone  3  at  765  Seconds 
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Figure  16:  Far-Field  correlation  of  a  Whale  Moan. 
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Sixteen-second  FFT  correlations  were  calculated  every  four  seconds  over  the  entire 
possible  range  of  correlations  for  each  whale  moan  in  the  first  data  set.  The  maximum 
magnitude  within  that  correlation  was  then  determined  and  saved.  The  maximum  values  for 
the  correlations  over  the  range  of  possible  correlations  were  plotted.  Figure  17  shows  a  plot 
of  the  maximum  correlations  of  Phone  3  to  Phone  1  at  the  543  second  time  offset  for  Phone 
1.  The  maximum  of  these  maximum-correlation  values  occurs  at  approximately  the  765 
second  time  offset  for  Phone  3,  as  shown  in  Figure  16.  This  verifies  the  correlation  chosen 
by  the  symbolic  correlation  program  for  this  particular  whale  moan. 
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Figure  17:  Maximum  Far-Field  Correlation  Magnitudes. 

In  the  majority  of  the  cases,  these  correlations  provided  verification  that  the 
symbolically  chosen  set  of  far-field  correlations  were  correct.  When  the  maximum 
correlation  versus  time  plot  was  presented,  there  were  sometimes  peaks  of  similar  size  that 
would  indicate  that  the  correlation  could  be  to  any  of  two  or  three  different  whale  moans. 
Figure  18  is  a  plot  of  maximum  far-field  correlations  that  have  an  ambiguous  best 
correlation. 
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Maximum  Magnitudes  of  all  Possible  Correlations  of  Phone  3  to  Phone  1  at  477  Seconds 
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Figure  18:  Maximum  Far-Field  Correlation  Magnitudes. 

The  FFT  correlations  validated  the  signal  shape  correlation  algorithm.  In  the  data  set 
shown  in  Figure  11(a),  there  are  29  far-field  whale-moan  correlations.  FFT  correlations 
were  calculated  for  each  of  the  moans  and  the  results  were  compared  to  the  results  of  the 
shape  correlation  program.  Of  the  29  shape  correlations,  there  were  19  moans  where  at  least 
one  FFT  correlation  between  Phones  1  and  2  and  Phones  3  and  4  had  a  maximum 
correlation  at  the  same  time  offset.  Twenty-five  of  the  29  moan  correlations  had  a  peak, 
corresponding  to  the  time  offset  chosen  by  the  shape  correlation  program,  in  the  FFT  far- 
field  correlation  magnitudes  which  was  among  the  top  three  peaks. 

There  were  occasional  errors  made  by  the  correlation  program.  The  program  did  not 
attempt  to  correlate  regions  between  far-field  hydrophones  if  a  near-field  correlation  did 
not  exist.  This  sometimes  caused  a  false  correlation  between  far-field  hydrophones.  False 
correlations  also  took  place  at  the  end  of  the  data  set.  The  whale  moans  from  the  lower  set 
of  hydrophones  consistently  correlated  to  moans  from  the  upper  set  of  hydrophones  with  a 
large  time  offset.  At  the  end  of  the  data  set,  the  moans  that  should  have  been  correlated  were 


52 


correlated  for  the  upper  hydrophones  and  there  were  moans  in  the  lower  set  of  hydrophones 
that  would  obviously  correlate  to  moans  from  the  upper  set  of  hydrophones  in  the  next  data 
set.  Without  the  next  data  set  available,  some  of  the  uncorrected  moans  from  the  lower  set 
of  hydrophones  correlated  moans  from  the  upper  set  of  hydrophones  that  should  have 
remained  uncorrected. 

Almost  all  of  the  valid  correlations  of  whale  moans  between  the  far-apart  hydrophones 
showed  a  consistent  time  offset  of  approximately  240  seconds.  This  is  near  the  limit  of  the 
possible  straight-line  travel  time  of  sound  between  these  two  hydrophone  pairs.  The  lower 
set  of  hydrophones  shown  in  Figure  1 1  is  south  of  the  upper  set  of  hydrophones.  This 
indicates  that  the  whale  moans  were  coming  from  the  south. 

When  performing  the  far-field  correlations  of  whale  moans,  the  whale  moans  often  had 
a  goodness-of-fit  value  of  over  0.7.  This  indicated  that  there  was  very  little  attenuation  of 
the  signal  over  the  approximate  335  km  distance  between  the  hydrophone  pairs.  It  also 
suggests  that  these  low  frequency  whale  moans  may  be  coming  from  distances  of  many 
hundreds  of  kilometers  south  of  the  southern  pair  of  hydrophones. 
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VIII.  CONCLUSION 

A.  INTRODUCTION 

The  goals  of  this  thesis  have  been  met.  The  computer  vision  model  properly  identifies 
t-phase  signals,  whale  moans,  and  calibration  signals.  The  correlation  program  properly 
correlates  whale  moans  between  near-field  and  far-field  hydrophones. 

These  programs  are  not  production  quality  programs.  They  are  written  in  prototype 
languages  and  improvements  could  be  made  in  the  data  structures  and  in  the  method  of 
execution.  However,  they  can  serve  as  prototype  programs  for  incorporation  of  the  ideas 
presented  into  other  programs. 

B.  ACHIEVEMENTS 

There  were  major  accomplishments  of  this  thesis  that  can  be  of  significant  benefit  to 
the  research  community.  The  first  is  that  a  method  has  been  developed  for  unambiguously 
identifying  t-phase  signals  received  at  submarine  hydrophones.  This  has  been  a  problem  for 
geophysicists  studying  these  signals.  The  other  major  accomplishment  of  this  thesis  is  that 
a  method  has  been  developed  for  correlating  signal  shapes  between  both  near-field  and  far- 
field  hydrophones.  This  method  is  particularly  applicable  to  the  whale  research  community. 

Geophysicists  studying  seismic  events  in  the  northeast  Pacific  ocean  basin  have  a  set 
of  software  tools  that  allow  them  to  scroll  hydrophone  data  across  the  screen  of  a  graphics 
workstation  so  that  they  can  look  for  t-phase  signals.  By  taking  the  ideas  from  this  thesis, 
they  can  add  some  modules  to  that  software  and  develop  a  reliable  t-phase  detector.  There 
is  also  the  possibility  of  using  the  identification  techniques  to  develop  an  acoustic  tsunami 
warning  system.  The  ideas  from  the  correlation  program  could  be  used  to  improve  the 
current  correlation  techniques. 

This  thesis  demonstrates  a  "proof-of-concept"  for  whale  researchers  that  are 
interested  in  remotely  tracking  the  migration  paths  of  large  open-ocean  baleen  whales. 
These  whales  are  extremely  difficult  to  monitor  using  traditional  survey  methods.  A 
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method  is  presented  here  for  automatically  identifying  whale  moans  and  correlating  them 
to  both  near-field  and  far-field  hydrophones. 

C.    WEAKNESSES 

The  algorithm  for  the  identification  program  turns  on  pixels  in  the  low  level  processing 
based  on  a  moving  threshold.  There  is  a  target  percentage  of  pixels  that  the  algorithm  wants 
turned  on  for  each  second  in  order  to  be  able  to  develop  distinct  regions.  This  is  sometimes 
a  detriment  when  there  is  both  an  intense  t-phase  signal  and  whale  moans  occurring 
simultaneously.  Often  the  whale  signals  lose  a  lot  of  their  shape  because  their  pixels  move 
to  the  t-phase  signal. 

In  order  for  a  program  to  effectively  work  with  this  data  in  a  production  fashion,  it  must 
process  it  in  a  real-time  mode.  The  identification  program  does  not  work  in  this  manner.  An 
arbitrarily  large  section  of  data  is  handled  at  one  time  by  the  program.  That  entire  section 
of  data  is  manipulated  several  times  during  the  low-level  vision-processing  phase  of  the 
program.  The  high-level  vision-processing  phase  also  performs  each  step  with  a  view  to  the 
entire  data  set.  The  real-time  quality  of  the  software  was  not  a  concern  during  this  thesis 
since  the  programs  were  not  intended  to  be  production  programs. 

Another  weakness  of  the  programs  written  for  this  thesis  is  their  speed.  It  takes 
approximately  2.5  hours  of  computation  time  to  produce  a  single  plot  containing  20 
minutes  of  data.  Since  the  programs  were  not  intended  as  production  programs,  speed  was 
not  a  concern.  The  incorporation  of  the  ideas  in  this  thesis  into  a  production  program  can 
be  done  without  a  significant  performance  penalty. 

Finally,  the  methods  used  to  do  the  symbolic  correlation  should  be  verified  by  use  of 
Fourier  transform  correlation  techniques.  These  techniques  would  also  give  a  much  more 
accurate  time  delay  so  that  lines-of-position  can  be  precisely  determined. 
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APPENDIX  A:  SAMPLE  OF  RESULTS  FROM 
IDENTIFICATION  PROGRAM 
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4E+00,6.492546576639387E+00) . 

best_line_fit ( 14 , 3 . 92  699E- 

01,7.6586511500  63783E+0  0,7.24  87  0710  9104  52  9E+01, 1.693  048404  634  05 

9E+00,4.098832939811693E+00) . 

best_line_f it (15,3. 92699E- 01,9. 725690848498098E- 

01, 1. 14 8590 07 64 31 55 1E+0 0,4. 2 8654 93 9 07 19 89 8E- 

01,2.125210346795603E-01) . 

best_line_fit (16 , 3 . 92  699E- 

0 1, 4. 5722 55 52 92 87 52E+00, 3. 32 074 1577 1222 68E+0 1,1. 42 109 93 92 4 107 06 

E+00,2.859531853424639E+00) . 

best_line_fit (17 , 0 . 0E+00 , 1 . 2E+00 , 2 . 0E+00 , 0 . 0E+00 , 0 . 0E+00 ) . 

best_line_f it (18, 3 . 92699E- 

01, 9. 2 55047 91004 003 7E+00, 1 . 113 07650 14  67 854E+02 , 2 . 569582 53  80  554  3 

E+00,1.017420361986031E+01) . 

best_line_fit (19 , 0 . 0E+00 , 5 . 0E-01 , 2 . 5E-01 , 0 . 0E+00 , 0 . 0E+00 ) . 

best_line_fit (20,0.0E+00, 6 . 666 666666 666666E- 

01, 6. 66 66666666 66666E-01, 0 . 0E+00 , 0 . 0E+00 ) . 

best_line_f it (21,3. 92699E- 

01,5.47  6144  54  9077222E+00,4.07031598193867  5E+01,1.48722  3  0832  8584 

7E+00,3.025641297288723E+00) . 

best_line_f it (22 , 3 . 92699E- 

01, 7. 71209704  5472777E+00,8.73  34  82  5168824  9  5E+01, 2. 04  981068403  3  93 

3E+00,6.826388759258818E+00) . 

best_line_f it (23,3. 92699E- 

01 ,4. 65126940533963 5E+00, 3. 3053 607402 66237E+01, 1.5210  6413  72  8622 

4E+00,3.204781893113417E+00) . 

best_line_f it (24,3. 92699E- 

01, 4. 88762 87024 347 14E+00, 3. 3 14 52 076662 906E+0 1,1.723  43  80  95  964227 

E+00,4.327691218210757E+00) . 

best_line_fit (2  5 , 0 . 0E+00 , 6 . 666 6666666666 66E- 

01,6.666666666666666E-01,0.0E+00,0.0E+00) . 

best_line_fit (26 , 0 . 0E+00 , 5 . 0E-01 , 2 . 5E-01 , 0 . 0E+00 , 0 . 0E+00 ) . 

best_line_fit (27 , 2 .748893E+00 , 1 . 323083 134483 586E+00 , 2 . 14  6587578 

914  547E+0  0,4.585569334063  93  6E-01,2.6412  65631099703E-01) . 

best_line_fit (28 , 0 . 0E+00 , 6 .399999999999999E- 

01, 5. 5999999999 9999 9E-01, 4. 800000000000011E-01, 2. 4E-01) . 

best_line_fit (29 , 7 . 853  98E- 

01,1.03  3  83440  0904  913E+01, 1 . 4  9  850127 517733 3E+02 , 3 . 4  54  53  04199  593  5 

8E+00,1.840361163311271E+01) . 

best_line_fit (30,0.0E+00,1.0E+00, 1 .25E+00 , 0 . 0E+0  0 , 0.0E+00) . 

best_line_fit (31 , 7 . 85398E- 

01, 6. 53 511165 00 63 27E+00, 6. 24 674 19 857867 08E+01, 2. 0  69 6773 3 9 90 63 18 

E+00, 6.002532424575429E+00) . 

horizontal_line(13, 35, 109, 120) . 
horizontal_line(18,30,159,170) . 
horizontal_line(57, 24,427, 445) . 
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whale(3,9, [15,14] ) . 
whale(4,10,  [17,  16] )  . 
whale (5, 12, [21,20,19] ) . 
whale (6, 13,  [22] )  . 
whale (7, 14,  [23]  )  . 
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possible_quake( 1,7,  [9]  )  . 
possible_guake (2 , 27 , [45] ) 
possible_quake (3 , 52 , [73 ] ) 
possible_quake(4, 86, [127] 
possible_quake(5, 89, [130] 
possible_guake (6,93, [134] 

far_quake(l, [9] ) . 
far_quake(2, [45] ) . 
far_quake(3,  [73] )  . 
far_quake(4, [127] ) . 
far_guake(5, [130] ) . 
far_quake(6, [134] ) . 
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APPENDIX  B:  PLOTS  PRODUCED  BY  THE 
CORRELATION  PROGRAM 
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APPENDIX  C:  PROLOG  CODE  FOR  THE 
IDENTIFICATION  PROGRAM 


/*  Incremental  visual  processing:  operates  line  by  line  on  a  file.  */ 

/*  Note:  to  avoid  counting  a  particular  region,  add  an  */ 

/*  "uninteresting_region"  fact  for  it--will  help  avoid  space  overflow.  */ 

:-  no_style_check (single_var) ,  unknown (X, fail )  . 

:-  dynamic  lastrowegensym/1 ,  region_name_change/2 ,  final_code/2 , 

uninteresting_region/l,  pixel/3, pixel/4. 

/*  SEEM  SEEM  SEEM  SEEM  SEEM  SEEM  SEEM  SEEM  SEEM  SEEM  SEEM  SEEM  SEEM  SEEM  */ 
:-  dynamic  lastwhalegensym/1 , 

group_success/0,  group/2,  groupgensym/1 ,  whale/3,  outer_pixel/3 . 
:-  asserta (average_lines (20) ) ,  asserta (threshold_inc (7 ) ) . 
/*  SEEM  SEEM  SEEM  */ 

:-  ensure_loaded (vision4 ) . 

demo  :-  demo ( input  file)  . 

demo(Infile)  :-  incrsum ( Inf ile, sumf ile) , 

newg (70 ) ,  incrxgrow (Inf ile, growf ile) , 

incrshapes ( f inalgrowpict , shapes file) , 

tell (xf inalgrowpict ) ,  xshow( f inalgrowpict ) ,  told, 

high_level . 

rawdemo  :-  rawdemo (input file) ,  do_lisp_regions . 
rawdemo (Inf ile)  :-  newg(20),  incrxgrow ( Inf ile, growf ile) , 

incrshapes ( f inalgrowpict , shapesf ile) , 

tell (xf inalgrowpict ) ,  xshow( f inalgrowpict ) ,  told, 

high_level . 

/*  Two  ways  to  show  a  picture  file:  full  data,  or  compressed  data  */ 

show(Infile)  :-  see(Infile),  show2 . 

show2  :-  not (at_end_of_f ile) ,  readlineclump (L) ,  printlist(L),  nl,  show2 . 

show2  :-  seen. 

xshow ( Inf ile)  :-  see(Infile),  xshow2 . 

xshow2  :-  not (at_end_of_f ile) ,  readlineclump (L) ,  xprintlist (L) ,  nl ,  xshow2 . 

xshow2  :-  seen. 

xprintlist (  [  ] )  . 

xprintlist ( [NIL] )  :-  number(N),  code62(N,A),  put(A),  !,  xprintlist (L) . 

xprintlist ( [I IL] )  :-write(I),  !,  xprintlist (L) . 

code6  2 (N,A)  :-  M  is  (N  mod  62) ,  M<10,  ! ,  A  is  M+4  8. 

code62 (N,A)  :-  M  is  (N  mod  62) ,  M<3  6,  ! ,  A  is  M+5  5. 

code62 (N,A)  :-  M  is  (N  mod  62) ,  ! ,  A  is  M+61 . 

incrsum(Inf ile,Outf ile)  :-  incrmap4 ( Inf ile, Outf ile, sum4 ) . 

incrgrad( Inf ile,Outf ile)  :-  incrmap9 ( Inf ile, Outf ile, gradientthreshold9) . 

incrgrow( Inf ile, Outf ile)  :- 

incrmapl ( Inf ile, tempgrowf ile, reverse_threshold) , 

incrmap9 (tempgrowf ile, tempgrowf ile2 ,deorphan) , 

incrclump (tempgrowf ile2, f inalgrowpict ) , 

pixel_convert (f inalgrowpict , temppixelf ile) , 

pixel_join ( Inf ile, Outf ile) . 

/*  General-purpose  line-by-line  2D  array  processing:  applies  the  given  */ 
/*  function  to  every  3  by  3  square  in  the  array,  outputs  array  of  results.  */ 
/*  Function  must  be  the  name  of  a  10-argument  (9-input)  function  predicate.  */ 
incrmap9 (Inf ile, Outf ile, Function)  :-  see (Infile) ,  tell (Outfile) , 

readlineclump (LI) ,  readlineclump (L2 ) ,  readlineclump (L3 ) , 

getamap9 (LI , L2, L3 , Function)  ,  !. 
getamap9 (LI, L2, L3 , Function)  :-  at_end_of_f ile, 

amapitem9 (LI, L2 , L3 , Function, GL) ,  write_line (GL) , 

seen,  told,  ! . 
getamap9 (LI, L2,L3, Function)  :-  amapitem9 (LI, L2 , L3 , Function, GL) , 

write_line(GL) ,  readlineclump (L4) ,  getamap9 (L2 ,L3 , L4 , Function) . 
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/*  Like  above,  but  applies  to  2  by  2  squares.  */ 

/*  Function  must  be  the  name  of  a  5-argument  (4-input)  function  predicate.  */ 

incrmap4 ( Infile, Outfile, Function)  :-  see(Infile),  tell (Out  file) , 

readlineclump (LI ) ,  readlineclump (L2 ) ,  getamap4 (LI , L2 , Function) ,  !. 
getamap4 (LI , L2 , Function)  :-  at_end_of_f ile, 

amapitem4 (LI , L2 , Function, GL) ,  write_line (GL) , 

seen,  told,  ! . 
getamap4 (LI , L2 , Function)  :-  amapitem4 (LI , L2 , Function, GL) , 

write_line (GL) ,  readlineclump (L3 ) ,  getamap4 (L2 , L3 , Function )  . 

/*  Like  above,  but  applies  to  single  cells.  */ 

/*  Function  must  be  the  name  of  a  2-argument  (1-input)  function  predicate.  */ 

incrmapl ( Infile, Outfile, Function)  :-  see(Infile),  tell (Out file) , 

readlineclump (LI ) ,  getamapl (LI , Function) ,  !. 
getamapl (LI , Function)  :-  at_end_of_f ile, 

amapiteml (LI, Function, GL) ,  write_line (GL) , 

seen,  told,  ! . 
getamapl (LI , Function)  :-  amapiteml (LI , Function, GL) , 

write_line (GL) ,  readlineclump (L2 ) ,  getamapl (L2 , Function)  . 

/*  SEEM  SEEM  SEEM  SEEM  SEEM  SEEM  SEEM  SEEM  SEEM  SEEM  SEEM  SEEM  SEEM  SEEM  */ 
/*  STRT  STRT  STRT  STRT  STRT  STRT  STRT  STRT  STRT  STRT  STRT  STRT  STRT  STRT  */ 
/*  Like  above  but  with  dynamic  threshold  */ 
incrxgrowf Infile, Outfile)  :- 

incrxmapl (Infile, tempgrowf ile, reverse_x_threshold) , 

incrmap9 (tempgrowf ile, tempgrowf ile2 ,deorphan) , 

incrf ill ( tempgrowf ile2 , tempgrowf ile3) , 

get_lines (tempgrowf ile3 , tempgrowf ile4 ) , 

remove_manmade_signals ( tempgrowf ile4, tempgrowf ile5) , 

get_lines ( tempgrowf ile5, tempgrowf ile6) , 

remove_manmade_signals ( tempgrowf ile6, tempgrowf ile7 ) , 

diagincrclump (tempgrowf ile7 , f inalgrowpict ) , 

pixel_convert (f inalgrowpict, temppixelf ile) , 

pixel_join( Infile,  Outfile)  , 

get_pixel_values ( Infile) . 
incrxmapl (Infile, Outfile, Function)  :-  see(Infile),  tell (Outfile) , 

abolish (last_line/l ) ,  abolish (intensity/ 4 ) ,  assert a ( last_line (1 ) ) , 

readlineclump (LI ) ,  getamapxl (LI , Function) ,  !. 
getamapxl (LI, Function)  :-  at_end_of_f ile,  last_line (Line) , 

gradient_threshold(G) ,  assert z ( intensity (Line, 0 ,G, 0) ) , 

amapiteml (LI , Function, GL) ,  write_line (GL) ,  seen,  told,  !. 
getamapxl (LI, Function)  :-  last_line (Line) ,  gradient_threshold (G) , 

assertzf intensity (Line, 0,G,0) ) ,  amapiteml (LI, Function, GL) , 

get_average_intensity (Line) ,  abolish (iteration/1) ,  asserta (iteration (0) ) , 

getamapx2 (LI, Function, Line,GL,NewGL) , 

write_line (NewGL) ,  readlineclump (L2 ) ,  NewLine  is  Line  +  1, 

retract (last_line (Line) ) ,  asserta (last_line (NewLine) ) , 

getamapxl (L2, Function) . 
getamapx2 (LI, Function, Line, GL,GL)  :-  iteration (6 ) . 
getamapx2 (LI , Function, Line, GL,GL)  :-  intensity (Line, X, G, I ) , 

X  <  15,  X  >  10. 
getamapx2 (LI, Function, Line, GL,GL)  :-  intensity (Line, X, G, I ) , 

X  >  14,  G  >  200. 
getamapx2 (LI, Function, Line, GL,GL)  :-  intensity (Line, X, G, I ) , 

X  <  11,  G  <  36. 
getamapx2 (LI, Function, Line, GL, NewGL)  :-  intensity (Line, X, G, I ) ,  X  >  14, 

NewG  is  G  +  X  /  2,  newt(NewG),  retract ( intensity (Line, X,G, I )) , 

assert z (intensity (Line, 0,NewG,0) ) ,  amapiteml (LI , Function, GL2) , 

get_average_intensity (Line) ,  retract (iteration (Loop) ) ,  NewLoop  is  Loop  +  1, 

asserta (iteration (NewLoop) ) ,  getamapx2 (LI , Function, Line,GL2 , NewGL) . 
getamapx2 (LI , Function, Line, GL, NewGL)  :-  intensity (Line, X,G, I ) ,  X  <  11, 

NewG  is  G  -  X  /  2,  newt (NewG),  retract ( intensity (Line, X,G, I )) , 

assertz (intensity (Line, 0,NewG, 0 ) ) ,  amapiteml (LI , Function, GL2) , 

get_average_intensity (Line) ,  retract ( iteration (Loop) ) ,  NewLoop  is  Loop  +  1, 

asserta (iteration (NewLoop) ) ,  getamapx2 (LI , Function, Line, GL2 , NewGL) . 

get_average_intensity (Line)  :-  intensity (Line, N,G, I) ,  N  >  0, 
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retract (intensity (Line, N, G, I )) ,  Avgl  is  I  /  N,  IntAvgl  is  integer (Avgl ) , 
assertzl  intensity (Line, N,  G, IntAvgl ) ) . 
get_average_intensity (Line) . 

reverse_x_threshold (A, * )  :-  gradient_threshold (G) ,  A<G, ! . 
reverse_x_threshold (A, 0 )  :-  last_line (Line) ,  retract ( intensity (Line, X,G, I 
Newl  is  I  +  A,  Xpl  is  X+l,  !,  assertz ( intensity (Line, Xpl , G, Newl )) ,  !. 


/*  Fill  in  gaps  across  rows,  columns,  and  diagonals 
incrf ill (Inf ile,Outfile)  :-  see(Infile),  tell (Outfi 

readlineclump ( [L2a I L2 ] ) ,  readlineclump (L3 ) ,  write_ 

incrfill2 (LI, [L2alL2] ,L3,L3, [L2a] ) ,  seen,  told,  !. 
incrf i 112 (LI , L2 ,_, L3 , L2new2)  :-  at_end_of_f ile,  wri 

wri te_line (L3 ) ,  !. 
incrf i 112 (LI, [_, L2b, L2c] , L3 , [L3alL3orig] ,L2newl)  :- 

append (L2 newl , [L2b,L2c] ,L2new2) ,  write_line (L2new2 

incrfill2 (L2new2, [L3alL3orig] , L4 , L4 , [L3a] ) . 
incrf i 112 ( [_IL1] , [_, 0 IL2] , [_IL3] , L3orig, L2newl )  :- 

incrf il 12 (LI, [0 IL2] , L3 , L3orig, L2new2 ) . 
incrf ill  2 ( [_,Llb,0 I  LI] ,  [_IL2] ,  [0 IL3] , L3orig, L2newl ) 

append (L2 newl,  [0] ,L2new2) ,  incrf ill2 ( [Lib, 0  I  LI ] , L2 
incrfill2 ( [0 IL1] , [_IL2] , [_,L3b,0 IL3] , L3orig, L2newl ) 

append (L2newl, [0] ,L2new2) ,  incrfill2(Ll,L2,[L3b,0| 
incrf i 112 ( [_, 0  I  LI] ,  [_| L2 ] ,  [_, 0 |L3 ] , L3orig, L2newl )  : 

incrf ill2 ( [0IL1] ,L2, [0IL3] , L3orig, L2new2 ) . 
incrf ill  2 ( [_IL1] ,  [0,_,0|L2] ,  [_IL3] , L3orig, L2newl )  : 

incrf i 112 (LI,  [0,0  I L2] , L3 , L3orig, L2new2 )  . 
incrfill2 ( [_IL1] , [_, * IL2] , [_IL3] , L3orig, L2newl )  :- 

incrf i 112 (LI, [* IL2] , L3 , L3orig, L2new2 ) . 


to  make  fuller  shapes  */ 
le) ,  readlineclump (LI ) , 
line(Ll) , 

te_line (L2) , 


),  readlineclump (L4 ) , 
append (L2newl , [0 ] , L2new2 ) , 

, L3 , L3orig, L2new2 ) . 

L3 ] , L3orig, L2new2 ) . 

-  append (L2newl , [0 ], L2new2 

-  append (L2newl , [0 ], L2new2 
append (L2newl , [ * ] , L2new2 ) , 


/*  Find  man-made  signals  and  mark  them  uniquely.  */ 
get_lines ( Inf ile,Outf ile)  :-  see(Infile),  tell (Outf ile) , 

readlineclump (L2 ) ,  readlineclump (L3 ) ,  get_cal_signal (L2 

get_cal_signal (L3 , L3s) ,  write_line (LI ) , 

get_lines2 (Ll,L2s,L3s,L3s, [ ] ) ,  seen,  told,  !. 
get_lines2 (LI , L2 , L3old, L3 , L2new)  :-  at_end_of_f ile,  write_line (L2 

write_line (L3 ) ,  ! . 
get_lines2 ( [*,m,m] , [*,0,0] , L3 , L3orig, L2newl )  :- 

append (L2 newl , [ * ,m,m] , L2new2) ,  get_lone_pixels (L2new2 , [ ] , L2new3 ) 

write_line (L2new3 ) ,  readlineclump (L4 ) , 

get_cal_signal (L4 , L4s) ,  get_lines2 (L2new2 , L3orig, L4s, L4s, [ ] ) . 
get_lines2 (L1,[*,0,0],[*,0,0] , L3orig, L2newl )  :- 

append (L2 newl , [ * ,m,m] , L2new2) ,  get_lone_pixels (L2new2 , [ ) , L2new3 ) 

write_line (L2new3 ) ,  readlineclump (L4 ) , 

get_cal_signal (L4 , L4s) ,  get_lines2 (L2new2 ,L3orig,L4s,L4s, [] ) . 
get_lines2 (LI, [ L2a, L2b, L2c] ,L3, L3orig, L2newl )  :- 

append ( L2newl ,  [L2a,  L2b, L2c] ,L2new2) ,  get_lone_pixels (L2new2 , 

write_line (L2new3 ) ,  readlineclump (L4 ) , 

get_cal_signal (L4 , L4s) ,  get_lines2 (L2new2 , L3orig, L4s, L4s, L3a 


readlineclump (LI 
L2s)  , 


L2new3 ) , 


LI 


get_lines2 ( 
append (L2new 
get_lines2 ( [ 

get_lines2 ( [m 
append (L2new 
get_lines2 ( [ 

get_lines2 ( [* 
append (L2 new 
get_lines2 ( [ 

get_lines2 ( [* 
append (L2 new 
get_lines2 ( ( 

get_lines2 ( [* 
append (L2new 
get_lines2 ( ( 

get_lines2 ( [_ 
append (L2new 
get_lines2 ( [Lid 

get_lines2 ( [_|L1 


IL2 


L3] , L3orig, L2new) 


rn , 


L3orig, L2new2 ) . 
IL2] ,  [*,*,*,* I L3 


L3orig, L2new) 


L3orig, L2new2 ) 


IL1] ,  [*,0,0, 

m] , L2new2 ) , 

[*IL2] , [*IL3] 
*,*,* I  LI]  ,  [*,0, 0, * 
[ * ,m,m] , L2new2 ) , 
I  LI ] ,  [*IL2] ,  [* IL3) 
*, *,mlLl] , [*,0,0, * IL2] 
[ * ,m,m] , L2new2 ) , 
I  LI] ,  [* IL2] ,  [* IL3] ,L3orig,L2new2)  . 

0,0,* IL1]  ,  [*,0,0, *|L2] , [_, ,L3dlL3] , L3orig, L2new) 

[ * ,m,m] , L2new2 ) , 

I  LI] ,  [* IL2] ,  [L3dlL3]  , L3orig, L2new2 )  . 

m,m, *  I  LI]  ,  [*,0,0,  *  I L2  ]  ,  [_,_,_, L3d I L3 

[ * ,m,m] , L2new2 ) , 

I  LI] ,  [* IL2] ,  [L3dlL3]  , L3orig, L2new2 )  . 

Lid  I  LI]  , [*,0,0, * IL2] , [*,0,0, * IL3] 

[*,m,m] ,L2new2) , 

LI] , [* IL2] , [* IL3] ,L3orig,L2new2) . 
, [L2alL2] , [_IL3] , L3orig, L2new)  :- 


L3] ,L3orig, L2new) 


, L3orig, L2new) 


L3orig, L2new) 


72 


append (L2 new, [L2a] , L2new2) ,  get_lines2 (LI , L2 , L3 , L3orig, L2new2 ) . 


/*  Find  the  characteristic  calibration  signal  and  change  signals  to  match.  */ 
get_cal_signal ( [ * , A, * , 


******* 


****** 


*  *  * 


*,*,*,*, B,C,0,0,D, *, 

********** 

*,'*|L],Ls)  •-' 

set_to_calsignal ( 

[*7\******** 
********** 

*,*,*/*#B,C,0,0,D,*, 

********** 

*!*IL]  ,  '[]  !ls)  '. 
get_cal_signal ( [ * , A, * , *  ,  * 
********** 

*,*,*,* ,B,C, 0,D, E, * , 
********** 

*!*IL] ,Ls)  :- 

set_to_calsignal ( 

f  *   A   *   ******* 
********** 


*  *   *   * 

*  *   *   * 

* ',  *  i  l  i , 


,C,0,D,E, *, 
***** 

,Ls)  . 


get_cal_signal ( [  * ,  A, *,*,*,*,*, 
********** 

*,*,*, *,B,C,D,0/E,*, 

********** 

*, * IL] ,Ls)  :- 
set_to_calsignal ( 

f*A******** 
I    ,  rt  ,    ,1,1,         I         It 

********** 

*,*/*/*#B,C,D,0,E,*, 

********** 

*!*1l] ,  i] !ls) ! 

get_cal_signal ([*, A,* , *,*,*,*, 

*  *  *  *  *  *  *  *_*  *. 

*  * 

*  * 


*  *  * 


*  *  * 


*  *  * 


B,C,0,0,D,*,*,*IL],Ls) 
set_to_calsignal ( 

\  *    h     ******** 
\     1 1\  i     i      i     i     i     i     i      t     i 

********** 
********* 


6,0,0,0,0, *, *, *  IL]  ,  [ ]  ,Ls)  . 
get_cal_signal (L, L)  . 

set_to_cal signal ( [ ] , Ls, Ls ) . 
set_to_cal signal ( [ * |L] , Ls, Lsnew)  : 

set_to_calsignal (L, Lsnew2 , Lsnew) . 
set_to_calsignal ( [0  I L] , Ls, Lsnew)  : 

set_to_calsignal (L, Lsnew2 , Lsnew) . 


append ( Ls , [ * ] , Lsnew2 ) 
append ( Ls ,  [ c ] , Lsnew2 ) 


/*  Get  pixels  that  have  no  upper  or  lower  neighbor  and  transform  into  an  m  */ 

/*  These  pixels  will  be  put  with  the  group  of  probable  man-made  pixels.  */ 

get_lone_pixels ( [ ] , Ls , Ls) . 

get_lone_pixels ( [ * , 0] , L, Ls)  :-  append ( L, [* ,m] , Ls ) . 

get_lone_pixels ( [ * , 0, *  I  Rest ] , L, Ls )  :-  append (L,  (*, m] , Lnew)  , 

get_lone_pixels ( [ *  I  Rest ] , Lnew, Ls)  . 
get_lone_pixels ([Ll|Rest],L,Ls)  :-  append (L, [LI ] , Lnew) , 

get_lone_pixels (Rest , Lnew, Ls) . 

/*  Remove  all  the  m  and  c  markings  from  a  file.  */ 
remove_manmade_signals ( Inf ile, Outf ile)  :-  see(Infile),  tell (Outf ile) , 
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readlineclump (LI ) ,  rernove_manmade_signals2 (LI ,X) ,  seen,  told,  !. 
remove_manmade_signals2 (LI, Llnew)  :-  at_end_of_f ile,  wri te_line (LI 
remove_manmade_signals2 ( [m] , LI )  :-  append (LI ,[*], Llnew 

readlineclump (L2 ) ,  remove_manmade_signals2 (L2 , []) . 
remove_manmade_signals2  (  [X]  ,  LI )  :-  append(Ll ,  [X]  ,  Llnew 

readlineclump (L2 ) ,  remove_manmade_signals2 (L2 , []) . 
remove_manmade_signals2 ( [ml  LI ] , Llnew) 

remove_manmade_signals2 (Ll,Llnewer) . 
remove_manmade_signals2 ( [c I  LI ] , Llnew) 

remove_rnanmade_9ignals2  (LI, Llnewer)  . 
remove_manmade_signals2 ( [XI  LI ] , Llnew) 

remove_manmade_signals2 (LI , Llnewer ) . 


-  append ( Llnew, 

-  append(Llnew, 

-  append (LI new, 


,  write_line (Llnew) 
,  write_line (Llnew) 
* ] , Llnewer) , 
* ] , Llnewer) , 
X] , Llnewer) , 


/*  Remove  pixels  that  have  no  upper  or  lower  neighbors.  */ 

/*  This  will  tend  to  get  rid  of  any  straight  (man-made)  lines  in  the  data.  */ 

remove_lone_pixels ( Inf ile, Out  file)  :-  see(Infile),  tell (Outf ile ) , 

readlineclump ( [X  I  LI ]) ,  remove_lone_pixels2 ( [X  I  LI ] ,  [X] ) ,  seen,  told,  !. 
remove_lone_pixels2 (LI , Llnew)  :-  at_end_of_f i le,  wri te_line ( LI ) ,  !. 
remove_lone_pixels2 ( [XI , X2 ] , LI )  :-  append (LI , [X2 ], Llnew) ,  wri te_l ine (Llnew) , 

readlineclump ( [X3 IL2] ) ,  remove_lone_pixels2 ( [X3 I L2 ] , [X3 ] ) . 
remove_lone_pixels2 ([*,0,*|L1], Llnew)  : -  append (LI new, [ * ] , Llnewer) , 

remove_lone_pixels2 ( [ * , *  I  LI ] , Llnewer)  . 
remove_lone_pixels2 ( [XI ,X2 I  LI ] , Llnew)  : -  append (Llnew,  (X2 ] , Llnewer) , 

remove_lone_pixels2 ( [X2 I  LI ] , Llnewer )  . 

get_pixel_values ( Inf ile)  :-  see(Infile),  readlineclump (LI ) , 

get_line_values (1 ,L1) , seen. 
get_line_values (Line, List )  :-  at_end_of_f ile, 

get_line_values2 (1 , Line, List ) . 
get_line_values (Line, List )  :-  get_line_values2 (1 , Line, List ) , 

NewLine  is  Line  +  1,  readlineclump (NewList ) , 

get_line_values (NewLine, NewList ) . 
get_line_values2 (Row, Line, [ ] ) . 
get_line_values2 (Row, Line,  [ First  I  Rest ]  )  :-  pixel (Row, Line, X) , 

retract (pixel (Row, Line, X) ) ,  assertfpixel (Row, Line, X, First ) ) , 

NewRow  is  Row  +  1,  get_line_values2 (NewRow, Line, Rest )  . 
get_line_values2 (Row, Line,  [First  I  Rest ] )  :-  NewRow  is  Row  +  1, 

get_line_values2 (NewRow, Line, Rest ) . 

/* 

look_for_vertical_line  :-  last_line (Line) ,  numzeros (Line, Zeros) , 
Lineml  is  Line  -  1,  numzeros (Lineml , ZerosLml ) ,  Dif  is  Zeros  -  ZerosLml , 
abs (Dif ,AbsDif ) ,  AbsDif  >  14,  assertz (vertical_line (Line)  )  . 

look_for_vertical_line . 

*/ 

/*  Group  nearby  regions  together.  Find  whales,  calibration  signals,  ...  */ 
high_level  :-  circumference,  get_group,  reorder_groups,  characteri ze_groups ( 1 
get_horizontal_lines,  get_vertices,  get_whale,  get_calibration_signals, 
get_earthquakes ,  get_unknowns ,  get_unknown_group_qualities , 
get_more_whales,  print_results . 

/*  Find  the  outer  pixels  in  each  region  and  assert  them.  */ 

/*  This  keeps  us  from  having  to  test  every  pixel  in  every  region  to  see 

if  two  regions  are  in  the  same  group.  */ 
circumference  :-  abolish (outer_pixel/3 ) ,  circumf erence2  (1 )  . 
circumf erence2 (Region)  :-  do_circumf erence (Region) ,  NewRegion  is  Region  +  1, 

area (NewRegion, X) ,  ci rcumference2 (NewRegion) . 
circumf erence2 (Region) . 
do_circumf erence (AreaNo)  :-  pixel (X, Y, AreaNo,_) ,  not ( inner_pixel (X, Y, AreaNo) ) 

assertz (outer_pixel (X, Y, AreaNo) ) ,  fail . 
do_circurnf  erence  (AreaNo)  . 
inner_pixel (X,Y, AreaNo)  :-  pixel (X, Y, AreaNo, _) , 

lef t (X, Y, AreaNo) ,  right (X, Y, AreaNo) ,  up (X, Y, AreaNo) ,  down (X, Y, AreaNo ) . 
lef t (X,Y, AreaNo)  :-  LeftCell  is  X-l,  pixel (Lef tCell , Y, AreaNo, _)  . 
right (X,Y, AreaNo)  :-  RightCell  is  X+l,  pixel (RightCell , Y, AreaNo, _) . 
up(X,Y, AreaNo)  :-  UpCell  is  Y  +  l,  pixel (X,UpCell , AreaNo, _)  . 
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down(X, Y,AreaNo)  :-  DownCell  is  Y-l,  pixel (X,DownCell , AreaNo,_) . 

/*  Group  nearby  regions  together  into  groups  of  regions.  */ 

get_group  :-  abolish (lastgroupgensym/1 ) ,  abolish (group/2 ) ,  get_group ( 1 ) . 

get_group (Region)  :-  get_group2 (Region) ,  NewRegion  is  Region  +  1, 

area (NewRegion, X) ,  get_group (NewRegion) . 
get_group (Region) . 
get_group2 (Rl )  :-  abolish (group_success/0 ) ,  lastgroupgensym (GroupID) , 

Gid  is  GroupID,  get_group3 (Rl , GroupID, Gid) , combine_regions (Rl , GroupID) . 
get_group2 (Rl )  :-  groupgensym (K) ,  assertz (group (K, [Rl ])) ,  get_group4 (Rl , K) , 

combine_regions (Rl, K) . 
get_group3 (Rl , GroupID, Gid)  :-  GroupID  <  Gid  +  3,  get_group4 (Rl , Gid) , 

group_success . 
get_group3  (Rl,  GroupID,  Gid)  :-  GroupID  <  Gid  +  3,  NewGid  is  Gid  -  1, 

get_group3 (Rl, GroupID,  NewGid). 
get_group4 (Rl , GroupID)  :-  group (GroupID, Gl ist ) ,  member (Rl , Glist ) ,  R2  is  Rl  + 

add_nearby_groups (Rl , R2 , Group ID) ,  asserta (group_success) . 

/*  If  a  region  is  near  a  group,  then  add  it  to  that  group.  */ 
add_nearby_groups (Rl, R2, GroupID)  :-  R2  >  Rl,  group (GroupID, Glist ) , 

not (member (R2, Glist) ) ,  nearby (R1,R2) ,  append ( [R2] , Glist , NewGl ist ) , 

retract (group (GroupID, Glist ) ) , assertz (group (GroupID, NewGl ist ) ) , 

R3  is  R2  -  1,  add_nearby_groups (Rl, R3, GroupID) ,! . 
add_nearby_groups (R1,R2, GroupID)  :  -  R2  >  Rl ,  R3  is  R2  -  1 , 

add_nearby_groups (Rl , R3 , GroupID) , ! . 
add_nearby_groups (Rl, R2 , GroupID) . 

/*  If  a  region  is  a  member  of  two  groups,  then  combine  the  two  groups.  */ 
combine_regions (Rl , Gl )  :-  G2  is  Gl  -  8,  combine_regions2 (Rl ,G1 , G2 ) . 
combine_regions2 (R1,G1,G2)  :-  G2  <  Gl ,  combine_groups (Rl , Gl , G2 ) ,  G3  is  G2  +  1 

combine_regions2 (Rl , Gl ,  G3 )  . 
combine_regions2 (Rl , Gl,  G2 ) . 
combine_groups(Rl,Gl,G2)  :-  Gl  =\=  G2  ,  group (Gl , Gllist ) ,  member (Rl , Gllist ) , 

group (G2,G2 list) ,  member (Rl ,G21ist) ,  union (Gllist, G2 list ,G3 list) , 

retract (group (Gl, Gllist) ) ,  retract (group (G2 ,G2 list ) ) , 

assertz  (group (Gl,G31ist))  . 
combine_groups (Rl , Gl , G2 ) . 

/*  Find  out  if  two  regions  are  close  enough  together  to  be  called  nearby.  */ 
nearby (R1,R2)  :-  outer_pixel (XI , Yl , Rl ) ,  outer_pixel (X2 , Y2 , R2 ) , 

Xdif  is  XI  -  X2,  Ydif  is  Yl  -  Y2,  Dist  is  Xdif  *  Xdif  +  Ydif  *  Ydif, 

Dist  <  13 . 

/*  Due  to  group  combining,  there  are  missing  numbers  in  the  groups.  */ 
/*  Need  to  have  a  straight  sequence  of  groups  for  later  processing.  */ 
reorder_groups  :-  lastgroupgensym (Gmax) ,  get_start_groups (Gl , G2 ,Gmax) , 

reorder  ,qroups2 (Gl,G2,Gmax) . 
reorder_groups2 (Gl,G2,Gmax)  :-  G2  <  Gmax  +  1,  group (G2 ,  G21 )  ,  Glpl  is  Gl  +  1, 
retract (group (G2,G21) ) ,  assertz (group (Glpl , G21 )) ,  G2pl  is  G2  +  1, 
get_nonempty_group (G2pl ,G3, Gmax) ,  reorder_groups2 (Glpl,G3 ,Gmax) . 
reorder_groups2 (Gl , G2 ,Gmax) . 

/*  Return  Gl  as  the  first  empty  group,  and  G2  as  the  first  non-empty  group  */ 
get_start_groups (Gl ,G2,Gmax)  :-  get_empty_group ( 1 , Glpl ) ,  Gl  is  Glpl  -  1, 
Glp2  is  Glpl  +  1,  get_nonempty_group (Glp2 ,G2,Gmax)  . 

/*  Return  next  valid  group  number  */ 


get_nonempty_group (Gl , Gl , Gmax) 
get_nonempty_group (Gl ,G1 ,Gmax) 
get_nonempty_group ( Gl , G2 , Gmax ) 


-  group(Gl ,G11) . 

-  Gl  >  Gmax. 

-  G3  is  Gl  +  1,  get_nonempty_group (G3 ,G2, Gmax) 


/*  Return  next  invalid  group  number  */ 

get_empty_group (Gl , Gl )  :-  not (group (Gl , Gil )) . 

get_empty_group (G1,G2 )  :-  G3  is  Gl  +  1,  get_empty_group (G3 , G2 ) . 

/*  Find  out  the  basic  parameters  for  each  group  and  assert  them.  */ 
characterize_groups (Group)  :-  group (Group, Gl ) ,  character_groups2 (Group) 
NewGroup  is  Group  +  1,  characterize_groups (NewGroup) . 
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characterize_groups (Group) . 

character_groups2 (Group)  :-  group (Group, Glist ) , 

character_groups3 (Glist , Xmin, Xmax, Ymin, Ymax, Area, IntDPct, IntAvgl ) , 

assertz (group_qualities (Group, Xmin, Xmax, Ymin, Ymax, Area, IntAvgl, IntDPct) ) . 
character_groups3 (Glist , Xmin, Xmax, Ymin, Ymax, Area, IntDPct, IntAvgl )  : - 

get_bounding_box (Glist, Xmin, Xmax, Ymin, Ymax) ,  get_area (Glist , Area) , 

get_density (Xmin, Xmax, Ymin, Ymax, Area, Density ) , 

IntDPct  is  integer (Density ) , 

get_total_intensity (Glist , Loudness) ,  Avgl  is  Loudness  /  Area, 

IntAvgl  is  integer (Avgl ) . 
get_bounding_box (Glist, Xmin, Xmax, Ymin, Ymax)  : -  get_xmin (Glist, Xmin) , 

get_xmax (Glist , Xmax) ,  get_ymin (Glist, Ymin) ,  get_ymax (Glist , Ymax) . 
get_xmin (Glist, Xmin)  :-  get_xmin2 (Glist , 99999, Xmin) . 
get_xmin2 ( [ ] , Xmin, Xmin) . 
get_xmin2 ( [G I  List ], Tempmin, Xmin)  :-  xmin(G,Min),  Min  <  Tempmin, 

get_xmin2 (List, Min, Xmin) . 
get_xmin2 ( [G I  List ], Tempmin, Xmin)  :-  get_xmin2 (List , Tempmin, Xmin)  . 
get_xmax( Glist, Xmax)  :-  get_xmax2 (Glist , 0 , Xmax) . 
get_xmax2 ( [ ] , Xmax, Xmax) . 
get_xmax2 ( [G I  List ] , Tempmax, Xmax)  :-  xmax(G,Max),  Max  >  Tempmax, 

get_xmax2 (List, Max, Xmax) . 
get_xmax2 ( [G I  List ], Tempmax, Xmax)  :-  get_xmax2 (List , Tempmax, Xmax)  . 
get_ymin (Glist, Ymin)  :-  get_ymin2 (Glist , 99999 , Ymin) . 
get^/min2 ( [ ] , Ymin, Ymin) . 
get_ymin2 ( [G I  List ], Tempmin, Ymin)  :-  ymin (G, Min),  Min  <  Tempmin, 

get_ymin2 (List , Min, Ymin) . 
get_ymin2 ( [G I  List ], Tempmin, Ymin)  :-  get_ymin2 (List , Tempmin , Ymin)  . 
get_ymax (Glist, Ymax)  :-  get_ymax2 (Glist , 0, Ymax) . 
get_ymax2 ( ( ] , Ymax, Ymax) . 
get_ymax2 ( [G I  List ], Tempmax, Ymax)  :-  ymax(G,Max),  Max  >  Tempmax, 

get_ymax2 (List , Max, Ymax) . 
get_ymax2 ( [G I  List ], Tempmax, Ymax)  :-  get_ymax2 (List , Tempmax, Ymax)  . 
get_area (Glist, Area )  :-  get_area2 (Glist, 0, Area) . 
get_area2 ( [ ] , Area, Area) . 
get_area2 ( [G I  List ] , OldArea, Area)  :-  area (G, PartArea) , 

NewArea  is  OldArea  +  PartArea,  get_area2 (List , NewArea , Area) . 
get_density (Xmin, Xmax, Ymin, Ymax, Area, Density)  : - 

Xdif  is  Xmax  -  Xmin  +  1,  Ydif  is  Ymax  -  Ymin  +  1,  BoxArea  is  Xdif  *  Ydif, 

Density  is  100.0  *  Area  /  BoxArea. 
get_total_intensity (Glist , Loudness)  :-  abolish (loudness/1 ) , 

asserta (loudness (0) ) ,  get_total_intensity2 (Glist) , loudness (Loudness) . 
get_total_intensity2 ( [ ] ) . 
get_total_intensity2 ( [R I L] )  :-  pixel (X, Y, R, Loud) ,  retract (loudness (Loudness) 

NewLoud  is  Loud  +  Loudness,  asserta (loudness (NewLoud) ) ,  fail. 
get_total_intensity2 ( [R I L] )  :-  get_total_intensity2 (L) . 

/*  Find  all  of  the  horizontal  lines  in  the  data  set  */ 

get_horizontal_lines  :-  abolish (horizontal_line/4 ) ,  get_horizontal_lines ( 1 ) . 

get_horizontal_lines (Group)  :-  group (Group, Glist ) , 

add_group_f ield_to_outer_pixels (Group, Glist ) , 

get_horizontal_lines2 (Group) , 

NewGroup  is  Group  +  1,  get_horizontal_lines (NewGroup) . 
get_horizontal_lines (Group) . 
get_horizontal_lines2 (Group)  :- 

bagof (  (X,Ylist),  bagof (Y, R"outer_pixel (X, Y, R, Group) , Ylist ) ,  XYlist), 

get_horizontal_lines3 (Group, XYlist) . 

get_horizontal_lines2 (Group, List) . 
get_horizontal_lines3 (Group, [ ] ) . 
get_horizontal_lines3 (Group, [ (X, Ylist) I XYlist] )  :-  length (Ylist, Ylen) , 

Ylen  >  8,  min (Yl ist, Ymin) ,  max (Ylist , Ymax) ,  Ydif  is  Ymax  -  Ymin  +  1, 

Density  is  Ylen  /  Ydif,  Density  >  0.5, 

assertz (horizontal_line (Group, X, Ymin, Ymax)  ) , 

get_horizontal_lines3 (Group, XYlist ) . 
get_horizontal_lines3 (Group, ((X, Ylist) I XYlist])  :  - 

get_horizontal_lines3 (Group, XYlist) . 

/*  Take  the  3  argument  outer_pixel  facts  and  associate  a  group  with  them  */ 
add_group_field_to_outer_pixels (Group, [] ) . 
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add_group_field_to_outer_pixels (Group,  [Region  I Glist ] )  :- 

change_outer_pixels (Group, Region) , 

add_group_f ield_to_outer_pixels (Group, Gl ist ) . 
change_outer_pixels (Group, Region)  :-  outer_pixel (X, Y, Region) , 

retract (outer_pixel (X, Y, Region) ) ,  assertz (outer_pixel (X, Y, Region, Group) ) , 

fail. 
change_outer_pixels (Group, Region) . 

/*  Find  intersections  between  horizontal  and  vertical  lines  and  assert  vertex.*/ 

get_vertices  :-  get_verticesl ,  remove_duplicate_vertices  . 

get_verticesl  :-  vertical_line ( Y) ,  get_vertices2 ( Y) ,  retract (vertical_line ( Y) ) , 

get_vertical_edge_regions (Y, Rlist ) ,  assertz (vertical_edge (Y,Rlist)),fail. 
get_verticesl . 
get_vertices2 (Y)  :-  hori zontal_line (Group, X, Ymin, Ymax) ,  Ydif  is  Ymax  -  Y, 

abs (Ydif ,AbsYdif ) ,  AbsYdif  <  5,  group (Group, Glist ) , 

not (vertex (Group, Glist, X, Y) ) ,  assertz (vertex (Group, Glist, X, Y) ) ,  fail. 
get_vertices2 ( Y )  :-  horizontal_line (Group, X, Ymin, Ymax) ,  Ydif  is  Ymin  -  Y, 

abs (Ydif , AbsYdif ) ,  AbsYdif  <  5,  group (Group, Glist )  , 

not (vertex (Group, Glist, X, Y) ) ,  assertz (vertex (Group, Glist, X, Y)  )  ,  fail. 
get_vertices2 ( Y)  :-  horizontal_line (Group, X, Ymin, Ymax) ,  Y  >=  Ymin,  Y  =<  Ymax, 

group (Group, Glist ) ,  not (vertex (Group, Glist , X, Y) ) , 

assertz (vertex (Group, Glist, X, Y) ) ,  fail. 
get_vertices2 (Y) . 

/*  Take  the  vertical  edge  and  associate  it  with  an  appropriate  region  */ 
get_vertical_edge_regions (Y, Rlist )  : - 
setof(R,  Group~X~outer_pixel (X, Y,R, Group) ,  Rlist). 

/*  There  may  be  several  Y  intercepts  for  the  X  lines.  Just  take  nearest.  */ 

remove_duplicate_vertices  :-  vertex (Group, Glist , X, Yl ) ,  vertex (Group, Glist , X, Y2 ) 
Y2  =\=  Yl,  horizontal_line (Group, X, Ymin, Ymax) ,  Ydifl  is  Ymax  -  Yl , 
abs(Ydifl,AbsYdifl) ,  AbsYdifl  <  5,  Ydif2  is  Ymax  -  Y2  ,  abs (Ydif 2, AbsYdi f 2 ) , 
AbsYdifl  <  AbsYdif2,  retract (vertex(Group, Glist ,  X,  Y2 )) ,  fail. 

remove_duplicate_vertices  :-  vertex (Group, Glist , X, Yl ) ,  vertex (Group, Glist , X, Y2 ) 
Y2  =\=  Yl,  horizontal_line (Group, X, Ymin, Ymax) ,  Ydifl  is  Ymax  -  Yl, 
abs (Ydifl, AbsYdif 1) ,  AbsYdifl  <  5,  Ydif2  is  Ymax  -  Y2  ,  abs (Ydif 2 , AbsYdi f 2  )  , 
AbsYdifl  >=  AbsYdif2,  retract (vertex (Group, Glist ,  X,  Yl ))  ,  fail. 

remove_duplicate_vertices  :-  vertex (Group, Glist , X, Yl ) ,  vertex (Group, Glist , X, Y2 ) 
Y2  =  \  =  Yl,  horizontal_line (Group, X, Ymin, Ymax) ,  Ydifl  is  Ymin  -  Yl, 
abs (Ydifl, AbsYdif 1) ,  AbsYdifl  <  5,  Ydif2  is  Ymin  -  Y2  ,  abs (Ydif 2 , AbsYdif 2 ) , 
AbsYdifl  <  AbsYdif 2,  retract (vertex(Group, Glist ,  X,  Y2 ))  ,  fail. 

remove_duplicate_vertices  :-  vertex (Group, Glist , X, Yl ) ,  vertex (Group, Glist , X, Y2 ) 
Y2  =\=  Yl,  horizontal_line (Group, X, Ymin, Ymax) ,  Ydifl  is  Ymin  -  Yl , 
abs (Ydifl, AbsYdif 1) ,  AbsYdifl  <  5,  Ydif2  is  Ymax  -  Y2  ,  abs (Ydif 2 , AbsYdi f 2 ) , 
AbsYdifl  >=  AbsYdif 2,  retract (vertex (Group, Glist ,X,  Yl ))  ,  fail. 

remove_duplicate_vertices . 

/*  Find  all  the  whale  sounds  in  the  data  set.  */ 

get_whale  :-  abolish (lastwhalegensym/1 ) ,  abolish (whale/3 ) ,  get_whale (1 ) . 

get_whale (Group)  :-  group (Group, Glist ) ,  get_whale2 (Group) , 

NewGroup  is  Group  +  1,  get_whale (NewGroup)  . 
get_whale (Group) . 
get_whale2 (Group)  :- 

group_qualities  (Group, Xmin, Xmax,  Ymin,  Ymax,  Area,  Loudness, Density )  , 

Xmin  >  20,  Ylength  is  Ymax  -  Ymin,  Ylength  >  12,  Ylength  <  60,  Area  >  50, 

Area  <  500,  Density  >  8,  Density  <  75,  whalegensym(K) , 

group (Group, Glist) ,  assertz (whale (K, Group, Glist )  )  . 
get_whale2 (Group) . 

/*  Find  the  calibration  signals  in  the  data  set.  */ 

/*  Calibration  signals  are  listed  by  time  (YMIN, YMAX),  instead  of  by  regions.*/ 
get_calibration_signals  :-  abolish (low_calibration_signal/l ) , 
abolish (overtone_region/3 ) ,  abolish (overtone_pixel/4) , 
abolish (calibration_signal/4) ,  get_low_calibration,  get_overtone . 

/*  The  calibration  signals  are  found  by  keying  on  the  lowest  tone.  */ 
get_low_calibration  :- 
group_qualities (Group, Xmin, Xmax, Ymin, Ymax, Area, Loudness, Density) , 
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Area  >  10,  Xinax  <  4,  group  (Group,  Glist )  , 
assertz (low_calibration_signal (Group, Glist ) ) ,  f ail . 
get_low_calibration . 

/*  The  overtone  signals  are  found  after  the  low  signal  is  identified.  */ 
get_overtone  :-  low_calibration_signal (LowGroup, Glist ) , 

get_overtone_regions (LowGroup) ,  get_overtone_groups (LowGroup) , 

eliminate_extraneous_pixels (LowGroup) ,  get_calibration_params (LowGroup) , 

fail. 
get_overtone. 
get_overtone_regions (LowGroup)  :  - 

group_qualities (LowGroup, Lxmin, Lxmax, Lymin, Lymax, Larea, Lloudness, Ldensity ) , 

ymin(R, Rymin) ,  Rymin  >  Lymin,  Rymin  <  Lymax, 

group (LowGroup, Glist) ,  not (overtone_region (LowGroup, Glist , R) ) , 

assertz (overtone_region (LowGroup, Glist, R) ) ,  fail. 
get_overtone_regions (LowGroup)  : - 

group_qualities (LowGroup, Lxmin, Lxmax, Lymin, Lymax, Larea, Lloudness, Ldensity  )  , 

ymax(R, Rymax) ,  Rymax  >  Lymin,  Rymax  <  Lymax, 

group (LowGroup, Glist) ,  not (overtone_region (LowGroup, Glist , R) ) , 

assertz (overtone_region (LowGroup, Glist, R) ) ,  f ail . 
get_overtone_regions (LowGroup)  : - 

group_qualities (LowGroup, Lxmin, Lxmax, Lymin, Lymax, Larea, Lloudness, Ldensity )  , 

ymax (R, Rymax) ,  Rymax  >  Lymax,  ymin(R, Rymin) ,  Rymin  <  Lymin, 

group (LowGroup, Glist) ,  not (overtone_region (LowGroup, Glist , R) ) , 

assertz (overtone_region (LowGroup, Glist , R) ) ,  f ail . 
get_overtone_regions (LowGroup)  : - 

group_qualities (LowGroup, Lxmin, Lxmax, Lymin, Lymax, Larea, Lloudness, Ldensity ) , 

ymax (R, Rymax) ,  Rymax  >  Lymax,  ymin (R, Rymin) ,  Rymin  <  Lymax, 

group (LowGroup, Glist) ,  not ( over tone_reg ion (LowGroup, R) ) , 

assertz (overtone_region (LowGroup, Glist , R) ) ,  fail. 
get_overtone_regions (LowGroup) . 
get_overtone_groups (LowGroup)  :-  overtone_region (LowGroup, LGlist, Region) , 

group (G, Glist ) ,  member (Region, Glist ) , 

not (overtone_group (LowGroup, LGlist, G, Glist) ) , 

assertz (overtone_group (LowGroup, LGlist, G, Glist)),  fail. 
get_overtone_groups (LowGroup) . 

/*  Sometimes  regions  extend  beyond  the  calibration  signal.  Pixels  from  these 

regions  must  be  eliminated  in  order  to  get  only  the  calibration  signal.  */ 
eliminate_extraneous_pixels (LowGroup)  : - 

overtone_group (LowGroup, LGList, OverGroup, OGList ) , 

cull_region_pixels (LowGroup, OGList) ,  fail . 
eliminate_extraneous_pixels (LowGroup) . 
cull_region_pixels (G, [] ) . 
cull_region_pixels (G, [R IList ] )  :-  cull_region_pixels2 (G, R) , 

cull_region_pixels (G, List ) . 
cull_region_pixels2 (G,R)  :-  pixel (X, Y, R, I ) ,  X  >  26,  X  <  29, 

assertz (overtone_pixel (X,Y,G,I)),  fail. 
cull_region_pixels2 (G,R)  :-  pixel (X, Y, R, I ) ,  X  >  50,  X  <  56, 

assertz (overtone_pixel (X,Y,G,I)),  fail. 
cull_region__pixels2  (G,R)  . 

/*  Find  out  when  the  calibration  signal  begins  and  ends.  */ 

get_cal i bra tion_pa rams (Group)  : -  get_overtone_params (Group, Oymin,Oymax) , 

get_calibration_ymin (Group, Cymin, Ymin) , 

get_calibration_ymax (Group, Oymax, Ymax) , 

group (Group, Glist) ,  assertz (calibration_signal (Group, Glist, Ymin, Ymax) ) . 
get_overtone_params (Group, Ymin, Ymax)  : - 

bagoffY,  I/sX"overtone_pixel  (X,  Y,  Group,  I )  ,  Ylist), 

min (Ylist , Ymin) ,  max( Ylist , Ymax) . 
get_calibration_ymin (Group, Oymin, Ymin)  : - 

group_qualities (Group, Lxmin, Lxmax, Lymin, Lymax, Larea, Lloudness, Ldensity) , 

Lymin  <  Oymin,  Ymin  is  Lymin. 
get_calibration_yrnin  (Group,  Oymin,  Ymin)  :-  Ymin  is  Oymin. 
get_calibration_ymax (Group, Oymax, Ymax)  : - 

group_qualities (Group, Lxmin, Lxmax, Lymin, Lymax, Larea, Lloudness, Ldensity) , 

Lymax  >  Oymax,  Ymax  is  Lymax. 
get_calibration_ymax (Group, Oymax, Ymax)  :-  Ymax  is  Oymax. 
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/*  Process  all  groups  to  see  if  they  might  be  earthquakes  */ 

get_earthquakes  :-  abolish ( lastquakegensym/1 )  ,  abolish ( lastf arquakegensym/1 ) , 

abolish (possible_quake/3 ) ,  abolish ( far_quake/2 ) , 

get_potential_quakes ( 1 ) ,  get_far_quakes (1 ) ,  def ine_f ar_quakes (1 ) , 

get_near_quakes . 
get_potential_quakes (Group)  :-  group (Group,Glist) ,  get_potential_quakes2 (Group) 

NewGroup  is  Group  +  1,  get_potential_quakes (NewGroup) . 
get_potential_quakes (Group) . 
get_potential_quakes2 (Group)  :- 

group_qualities (Group, Xmin, Xmax, Ymin, Ymax, Area, Loudness, Density ) , 

Xmin  <  10,  Area  >  5, 

not (calibration_signal (Group, _,_,_) ) ,  group (Group, Glist ) , quakegensym(K) , 

assertz (possible_quake (K, Group, Glist ) ) . 
get_potential_quakes2 (Group) . 

get_far_quakes (Quake)  :-  possible_quake (Quake, Group, Glist ) , 
group_qualities (Group, Xmin, Xmax, Ymin, Ymax, Area, Loudness, Density ) ,  Xmax  <  25, 
get_far_quakes2 (Quake) ,  NewQuake  is  Quake  +  1,  get_f ar_quakes (NewQuake) . 

get_far_quakes (Quake)  :-  possible_quake (Quake, Group, Glist ) , 
NewQuake  is  Quake  +  1,  get_f a r_quakes (NewQuake) . 

get_far_quakes (Quake) . 

get_far_quakes2 (Quake)  :-  combine_adjacent_far_quakes (Quake) . 

get_far_quakes2 (Quake)  :-  f arquakegensym (K) ,  assertz ( far_quake (K, [Quake] )) . 

get_near_quakes . 

combine_adjacent_far_quakes (Quake)  :-  lastf arquakegensym (K) ,  f ar_quake (K, Qlist ) 

Quakeml  is  Quake  -  1,  possible_quake (Quakeml , Groupml ,_) , 

member (Quakeml , Qlist) ,  possible_quake (Quake, G roup, _) , 

group_quali ties (Group,_,_, Ymin, Ymax, _,_,_) , 

group_qualities (Groupml ,_,_,_, Ymaxml, _,_,_) ,  ! , 

adjacent_quakes (Ymin, Ymax, Ymaxml ) , 

retract (f ar_quake (K, Qlist )) ,  assertz ( far_quake (K, [Quake IQlist] ) ) . 
adjacent_quakes (Ymin, Ymax, Ymaxml )  : - 

Ydif  is  Ymaxml  -  Ymin,  abs (Ydif , AbsYdif ) ,  AbsYdif  <  9. 
adjacent_quakes (Ymin, Ymax, Ymaxml )  :-  Ymaxml  >  Ymax. 

def ine_far_quakes (Quake)  :-  far_quake (Quake, Qlist ) , 

get_quake_regionlist (Qlist, [] , Rlist ) ,  retract (f ar_quake (Quake, Qlist ) ) , 

assertz (far_quake (Quake, Rlist )) ,  NewQuake  is  Quake  +  1, 

def ine_far_quakes (NewQuake) . 
def ine_far_quakes (Quake) . 
get_quake_regionl ist( [] , Rlist, Rlist) . 
get_quake_regionlist ( [Quake IQlist), [], Rlist)  : - 

possible_quake (Quake, Group, Glist) ,  get_quake_regionlist (Qlist, Glist, Rlist) . 
get_quake_regionlist ( [Quake IQlist ] , Reglist,  Rlist)  :  - 

possible_quake (Quake, Group, Glist ) , 

append (Glist, Reglist , Newlist) ,  get_quake_regionlist (Qlist ,Newlist , Rlist ) . 

/*  Assert  facts  for  all  groups  that  haven't  been  included  in  other  types  */ 
get_unknowns  :-  abolish (lastunknowngensym/1 ) ,  abolish (unknown_group/3 ) , 

get_unknownsl ( 1 ) . 
get_unknownsl (Group)  :-  group (Group, Glist ) , 

group_qualities (Group, _,_,_,_, Area, _,_)  ,  Area  >  7,  not (whale_member (Group) )  , 

not (possible_quake_member (Group) ) ,  not (overtone_group_member (Group) ) , 

unknowngensym(K) ,  assertz (unknown_group (K, Group, Glist ) ) , 

NewGroup  is  Group  +  1,  get_unknownsl (NewGroup) . 
get_unknownsl (Group)  :-  group (Group, Glist ) ,  NewGroup  is  Group  +  1, 

get_unknownsl (NewGroup) . 
get_unknownsl (Group) . 

whale_member (Group)  :-  whale (_, Group, _) . 


possible_quake_member (Group) 
overtone_group_member (Group) 
overtone_group_member (Group) 


-  possible_quake (_, Group, _) . 

-  overtone_group (_,_, Group, _) . 

-  overtone-group (Group, _,_,_) . 


/*  Find  the  angle  of  the  best  line  fit  through  a  group  */ 
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/*  Also  find  the  mean  distance  and  variation  of  the  pixels  in  the  group  from  */ 
/*  the  line.  Also,  find  the  mean  distance  and  variation  of  the  intersection  */ 
/*  of  the  perpendicular  to  each  pixel  with  the  line.  */ 
get_unknown_group_qualities  :-  abolish (best_line_f it/6 ) , 

get_uk_group_qualities ( 1 ) . 
get_uk_group_qualities (Ugroup)  : -  unknown_group (Ugroup, Group, Glist ) , 

get_uk_group_qualities2 (Ugroup) ,  NewGroup  is  Ugroup  +  1, 

get_uk_group_qualities (NewGroup) . 
get_uk_group_quali ties (Ugroup) . 
get_uk_group_qualities2 (Group)  :-  get_center (Group, Xc, Yc )  , 

assertz (best_line_f it (Group, 10,10,10,10,10)),  abolish (xdist/1) , 

abolish (xdistvariance/1 ) ,  abolish (dist_to_line/l ) , 

abolish (dist_to_line_variance/l ) , assertz (xdist (0) ) , assertz (xdistvariance (0) )  , 

assertz (dist_to_line (0) ) , assertz (dist_to_line_variance (0) )  , 

get_uk_group_qualities3 (Group,Xc, Yc, 0 . 0 )  , 

best_line_f it (Group, Angle, _,_, _,_) . 
get_uk_group_qualities3 (Group, Xc, Yc, Angle)  :-  Angle  >  2.75. 
get_uk_group_qualities3 (Group, Xc, Yc , Angle)  :  - 

get_line_eq (Xc, Yc , Angle, SinBeta ,CosBeta, P) , group (Group, Glist ) , 

get_di stances (Glist ,Xc, Yc , SinBeta, CosBeta, P) , get_distances2 (Group) , 

get_best_line_f it (Group, Angle) ,  NewAngle  is  Angle  +  0.392699, 

get_uk_group_qualities3 (Group, Xc, Yc , NewAngle) . 
get_di stances ( [ ] ,Xc,Yc, SinBeta, CosBeta, P) . 
get_di stances ( [Region  I  Group] , Xc, Yc, SinBeta, CosBeta, P)  :-  pixel (X, Y, Region, _)  , 

DistToLine  is  X  *  CosBeta  +  Y  *  SinBeta  -  P,  abs (DistToLine, AbsDistToLine) , 

DistToLineSQ  is  DistToLine  *  DistToLine,  Xdif  is  Xc  -  X,  Ydif  is  Yc  -  Y, 

DistToCenterSQ  is  Xdif  *  Xdif  +  Ydif  *  Ydif, 

XdistSQ  is  DistToCenterSQ  -  DistToLineSQ,  do_sqrt (XdistSQ, Xdist ) , 

retract (xdist (OldXdist) ) ,  NewXdist  is  OldXdist  +  Xdist, 

assertz (xdist (NewXdist) ) , 

retract (xdistvariance (OldXdistSQ) ) ,  NewXdistSQ  is  OldXdistSQ  +  XdistSQ, 

assertz (xdistvariance (NewXdistSQ) ) , 

retract (dist_to_line(OldDistToLine) ) , 

NewDistToLine  is  OldDistToLine  +  AbsDistToLine, 

assertz (dist_to_line (NewDistToLine) ) , 

retract (dist_to_line_variance (OldDistToLineSQ) ) , 

NewDistToLineSQ  is  OldDistToLineSQ  +  DistToLineSQ, 

assertz (dist_to_line_variance (NewDistToLineSQ) ) , 

fail. 
get_di stances ( [Region  I  Group] , Xc , Yc, SinBeta, CosBeta, P)  :  - 

get_di stances ( Group, Xc, Yc , SinBeta, CosBeta, P) . 
get_distances2 (Group)  : -  group_qualities ( Group, _, ,_, Area, ) , 

retract (xdist (OldXdist) ) ,  NewXdist  is  OldXdist  /  Area, 

assertz (xdist (NewXdist) ) , 

retract (xdistvariance (OldXdistVar) ) ,  NewXdistVar  is  OldXdistVar  /  Area, 

assertz (xdistvariance (NewXdistVar) ) , 

retract (dist_to_line (OldDistToLine) ) ,  NewDistToLine  is  OldDistToLine  /  Area, 

assertz (dist_to_l ine (NewDistToLine) ) , 

retract (dist_to_line_variance (OldDTLV) ) ,  NewDTLV  is  OldDTLV  /  Area, 

assertz (dist_to_l ine_variance (NewDTLV) ) . 
get_best_line_f i t (Group, NewAngle)  :  - 

best_line_f it (Group, Angle, Xdist, XdistVar, DistToLine, DistToLineVar ) , 

dist_to_line(NewDTL) ,  NewDTL  <  DistToLine, 

xdist (NewXdist) ,  xdistvariance (NewXDV) ,  dist_to_line_variance (NewDTLV) , 

retract (best_line_f it (Group, Angle, Xdist , XdistVar , DistToLine, DistToLineVar) ) , 

assertz (best_line_f it (Group, NewAngle, NewXdist, NewXDV, NewDTL, NewDTLV) ) . 
get_bes  t_l i  ne_f  i  t ( Group , NewAng 1 e )  . 
get_line_eq (Xc, Yc , Angle, SinBeta, CosBeta, P)  :-  acos (0 , PiOver2 ) , 

Beta  is  Angle  +  PiOver2,  sin (Beta , SinBeta ) ,  cos (Beta , CosBeta ) , 

P  is  Xc  *  CosBeta  +  Yc  *  SinBeta. 
do_sqrt (XdistSQ, Xdist)  :-  XdistSQ  >  0,  sqrt (XdistSQ, Xdist) ,! . 
do_sqrt (XdistSQ, Xdist)  :-  Xdist  is  0. 
get_center (Group, Xc , Yc)  :-  group (Group, Glist ) ,  abolish (xtotal /l ) , 

abolish (y total/1) ,  assertz (xtotal (0) ) , assertz (y total (0) ) , 

get_xy_totals (Glist) ,  group_qualities ( Group, _, ,_, Area,_,_) , 

xtotal (Xt),  ytotal(Yt),  Xc  is  Xt  /  Area,  Yc  is  Yt  /  Area. 
get_xy_totals  (  [  ] )  . 
get_xy_totals ( [Region  I Glist ] )  :-  pixel (X, Y, Region, _)  , xtotal (Xt) ,y total (Yt)  , 


80 


NewXt  is  Xt  +  X,  NewYt  is  Yt  +  Y,  retract  (xtotal  (Xt ))  ,  retract  (y  total  (Yt ))  , 
assertz (xtotal (NewXt ) ) ,  assertz (ytotal (NewYt )) ,  fail. 
get_xy_totals ( [Region IGlist] )  :-  get_xy_totals (Glist ) . 

/*  Take  unknown  groups  and  find  out  if  they  might  be  associated  with  whales.  */ 

get_more_whales  :-  get_more_whalesl ( 1 ) . 

get_more_whalesl (Wgroup)  :-  whale (Wgroup, Group, _) ,  get_more_whales2 (Group) , 

NewWgroup  is  Wgroup  +  1,  get_more_whalesl (NewWgroup )  . 
get_more_whalesl (Wgroup) . 
get_more_whales2 (Wgroup)  :- 

group_quali ties (Wgroup, WXmin, WXmax, WYmin, WYmax, WArea,_,_) , 

unknown_group (Ugroup, Group, Glist ) , 

group_quali ties (Group, UXmin, UXmax, UYmin, UYmax, UArea,_, _) , 

nearby _whale  (UXmin,  WXm in,  UXmax,  WXmax,  UYmin,  WYmin,  UYmax,  WYmax)  , 

retract (unknown_group (Ugroup, Group, Glist) ) , 

assert (associated_whale_group (Ugroup, Group, Glist ) ) , 

make_whale_group_list (Wgroup, Group) . 
get_more_whales2 (Wgroup) . 

/*  Determine  if  an  unknown  sound  is  near  enough  to  a  whale  sound  to  be  called 

a  part  of  the  same  sound.  */ 
nearby _whale (UXmin, WXmin, UXmax, WXmax, UYmin, WYmin, UYmax, WYmax)  : - 

XminMaxDif  is  UXmin  -  WXmax,  abs (XminMaxDif , AbsXminMaxDif ) , 

AbsXminMaxDif  <  14,  YminMaxDif  is  UYmin  -  WYmax, 

abs (YminMaxDi f , AbsYminMaxDif ) ,  AbsYminMaxDi f  <  8. 
nearby_whale (UXmin, WXmin, UXmax, WXmax, UYmin, WYmin, UYmax, WYmax)  : - 

XminMaxDif  is  WXmin  -  UXmax,  abs (XminMaxDi f , AbsXminMaxDif )  , 

AbsXminMaxDif  <  14,  YminMaxDif  is  WYmin  -  UYmax, 

abs (YminMaxDif , AbsYminMaxDif ) ,  AbsYminMaxDif  <  8. 

/*  Add  to  list  of  grouped  groups  of  whales  or  create  a  new  one  */ 
make_whale_group_li st (Wgroup, Group)  :-  whale_group (WhaleGroup) , 

member (Wgroup, WhaleGroup) ,  retract (whale_group (WhaleGroup) ) , 

append ( [Group] , WhaleGroup, NewWhaleGroup)  , 

assert (whale_group (NewWhaleGroup) ) . 
make_whale_group_list (Wgroup, Group)  :-  assert (whale_group ( [Wgroup, Group] ) ) . 

/* 

get_more_whalesl (Ugroup)  :-  unknown_group (Ugroup, Group, Glist) , 
group_qualities (Ugroup, Xmin,_,Ymin, Ymax, Area, _,_) ,  Xmin  >  10, 
Ymax  -  Ymin  >  4,  Area  >  20,  Area  <  240,  best_line_fit  (Ugroup, Angle, _,_,_,_)  , 
get_more_whales2 (Ugroup, Group, Angle) ,  NewUgroup  is  Ugroup  +  1, 
get_more_whalesl (NewUgroup) . 

get_more_whalesl (Ugroup)  :-  unknown_group (Ugroup, Group, Gl i st ) , 
NewUgroup  is  Ugroup  +  1,  get_more_whalesl (NewUgroup) . 

get_more_whalesl (Ugroup) . 

get_more_whales2 (Ugroup, Group, Angle)  :-  Angle  >  0.30,  Angle  <  1.2, 
whalegensym (K) ,  group (Group, Glist) ,  assertz (whale (K, Group, Glist ) ) , 
retract (unknown_group (Ugroup, Group,  Glist) )  , 
assertz (unknown_group (Ugroup, 0,  [0 ] ) )  . 

get_more_whales2 (Ugroup, Group, Angle) . 

*/ 


/* 

get_overlapping_whales  :-  get_overlapping_whalesl ( 1 )  . 

get_overlapping_whalesl (Ugroup)  :-  unknown_group (Ugroup, Group, Glist ) , 
Group  >  0, group_qualities (Ugroup, Xmin, _, Ymin, Ymax, Area, _,_) , 
Xmin  >  10,  Ymax  -  Ymin  >  50,  Area  >  239,  get_overlapping_whales2  (Ugroup) 
NewUgroup  is  Ugroup  +  1,  get_overlapping_whalesl (NewUgroup) . 

get_overlapping_whalesl (Ugroup)  :-  unknown_group (Ugroup, Group, Glist ) , 
NewUgroup  is  Ugroup  +  1,  get_more_whalesl (NewUgroup) . 

get_overlapping_whalesl (Ugroup) . 

get_overlapping_whales2 (Ugroup)  :-  group (Ugroup, Glist ) ,  member (R, Glist ) , 
delete (R, Glist , NewGlist) , 

*/ 
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/*  Generates  a  new  integer  each  time  it  is  called  */ 
whalegensym (K)  :-  retract ( lastwhalegensym (Kml )) ,  ! ,  K  is  Kml+1, 

assertz ( lastwhalegensym (K) ) . 
whalegensym (1 )  :-  assertz ( lastwhalegensym ( 1 )) . 

groupgensym (K)  :-  retract ( lastgroupgensym (Kml )) ,  ! ,  K  is  Kml+1, 

assertz (lastgroupgensym (K) )  . 
groupgensym (1 )  :-  assertz (lastgroupgensym! 1 )) . 

quakegensym (K)  :-  retract ( lastquakegensym (Kml )) ,  ! ,  K  is  Kml+1, 

assertz (lastquakegensym (K) ) . 
quakegensym ( 1 )  :-  assertz ( lastquakegensym ( 1 )) . 

f arquakegensym (K)  :-  retract (lastf arquakegensym(Kml )) ,  ! ,  K  is  Kml+1, 
assertz (lastf arquakegensym (K) ) . 
f arquakegensym ( 1 )  :-  assertz (lastf arquakegensym ( 1 )) . 

unknowngensym (K)  :-  retract (lastunknowngensym (Kml )) ,  ! ,  K  is  Kml+1, 

assertz (lastunknowngensym (K) ) . 
unknowngensym ( 1 )  :-  assertz ( lastunknowngensym ( 1 )) . 

print_results  :-  tell ( results) ,  listing (group) ,  listing (group_quali ties ) , 
listing (unknown_group) ,  listing (best_line_f i t ) ,  listing (horizontal_line) , 
listing (vertical_edge) ,  listing (vertex) ,  listing (calibration_signal ) , 
listing (overtone_region) ,  listing ( low_calibration_signal ) , 
listing (overtone_group) ,  listing (whale) ,  listing (possible_quake) , 
listing ( far_quake) ,  listing (numpixels ) ,  told. 

/*  Write  the  regions  of  size  >  5  to  a  file  in  lisp  readable  format.  */ 
do_lisp_regions  :-  tell ( lispf ile) ,  write_pixel_group (1 ) ,  told. 
write_pixel_group (G)  :-  group (G, Glist ) ,  already _wri tten (G) ,  Gpl  is  G  +  1, 

write_pixel_group (Gpl ) . 
write_pixel_group (G)  :-  group (G, Glist) ,  group_quali ties (G, _,_,_,_, Area, _,_) , 

Area  >  5,  write('(  '),  group_type (G,Type, NewGlist ) ,  write(Type),  write('  ') 

write_pixel_group2 (NewGlist ) , 

write(')'),  Gpl  is  G  +  1,  write_pixel_group (Gpl ) . 
write_pixel_group (G)  :-  group (G, Glist ) ,  Gpl  is  G  +  1,  wri te_pixel_group (Gpl ) 
write_pixel_group (G) . 
write_pixel_group2 ( [ ) ) . 
wri te_pixel_group2 ( [Region  I Glist ] )  :-  write_pixel_group3 (Region) , 

wri te_pixel_group2 (Glist )  . 
wri te_pixel_group3 (Region)  :-  pixel (Y, X, Region, _) ,  write('('), 

write(X),  write ('  '),  write(Y),  write(')'),  nl,  fail. 
write_pixel_group3 (Region) . 

/*  Get  the  group  type  for  a  group  and  associate  it  with  a  letter.  */ 
group_type (Group, 'w' , NewGlist )  :-  whale (_, Group, _) ,  whale_group (WhaleGroup) , 
member (Group, WhaleGroup) ,  get_whalegroup_regions (WhaleGroup, [], NewGlist) . 


group_type (Group, 'w' , Glist) 
group_type (Group, 'u' , Glist) 
group_type (Group, 'q' , Glist) 
group_type (Group, 'o' , Glist) 


-  whale (_, Group, Glist ) . 

-  unknown_group (_, Group, Glist ) . 

-  possible_quake (_, Group,  Glist )  . 

-  group (Group, Glist ) . 


/*  Find  all  the  regions  that  belong  to  one  whale  song  and  list  them.  */ 

get_whalegroup_regions ( [ ] ,WhaleList , WhaleList ) . 

get_whalegroup_regions ( [Group IGroupList ] , RegionList , WhaleList )  : - 
group (Group, Rl i st ) , 

append (Rl is t, RegionList , NewRegionList ) ,  assert (already_written (Group) ) , 
get_whalegroup_regions (GroupList, NewRegionList , WhaleList ) . 

/*  END  END  END  END  END  END  END  END  END  END  END  END  END  END  */ 

/*  SEEM  SEEM  SEEM  SEEM  SEEM  SEEM  SEEM  SEEM  SEEM  SEEM  SEEM  SEEM  SEEM  SEEM  */ 

/*  Region  "clumping"  using  the  gradient-threshold  array  as  input.  */ 
/*  This  version  more  appropriate  for  sound  processing.  */ 

incrclump(Infile,Outfile)  :-  abolish (region_name_change/2 ) , 

82 


abolish (lastrowegensym/1 ) ,  abolish ( final_code/2 ) , 

see(Infile),  tell( tempo lumpf ile) , 

readlineclump (GL1) ,  length (GL1 , N) ,  constant_vector ( * , N,CV) , 

incrclumprow (CV,CV,GL1) , 

see ( tempclumpf ile) ,  tell (Outfile) , 

abolish (lastrowegensym/1 ) ,  f ix_region_names ,  seen,  told, 
incrclumprow (RL1 , GL1 , GL2 )  :- 

a  t_end_o  f _f  i 1 e , 

incrclumprow2 (RL1 , *,  [* IGL1] ,  [ *  I GL2 ]  ,RL2) ,  wri te_line (RL2 )  , 

seen,  told,  !  . 
incrclumprow (RL1 , GL1 , GL2 )  :- 

incrclumprow2 (RL1 , * ,  [ *  I GL1 ] ,  I  *  I GL2 ]  , RL2 ) , 

wri te_l ine (RL2 ) ,  readlineclump (GL3 ) ,  incrclumprow (RL2 , GL2 , GL3 ) . 
/*  Reads  the  first  pass  output  back  in,  changes  the  region  names,  writes  out  */ 
f ix_region_names  :-  not (at_end_of_f ile) ,  readlineclump (L) , 

f ix_region_names (L, NL) ,  write_line (NL) ,  !,  f ix_region_names . 
f ix_region_names  :-  abolish ( region_name_change/2 ) ,  abolish ( final_code/2 ) . 
f ix_region_names ([],[]). 

f ix_region_names ( [ *  I L] ,  [ *  I NL] )  :-  !,  f ix_region_names (L, NL) ,  !. 
f ix_region_names ( [Rl I L] ,  [R2 I NL] )  : -  trans  it ive_region_name_change (Rl , R3 ) , 

xf inal_code (R3 , R2 ) ,  !,  f ix_region_names (L, NL) ,  !. 
f ix_region_names ( fRl IL] , (R2 INL] )  :-  xf inal_code (Rl , R2 ) , 

f ix_region_names (L, NL) ,  !. 
xfinal_code(Rl,R2)  :-  f inal_code (Rl , R2 ) ,  !. 
xf inal_code (Rl , R2 )  :-  rowegensym (R2 ) ,  assertz ( f inal_code (Rl , R2 ) ) ,  !. 

/*  Handles  a  single  row  during  incremental  clumping.  */ 

/*  First  arg  is  region  labels  for  previous  row,  second  arg  is  region  */ 

/*  label  for  cell  to  the  left  of  the  current  cell,  third  arg  is  list  */ 

/*  of  Os  and  *s  (gradient  threshold  output)  for  row  above,  fourth  arg  */ 

/*  is  same  for  current  row,  and  fifth  arg  is  output,  the  new  region  labels.  */ 

/*  At  end  of  row?  */ 

incrclumprow2 ( [ ],_,[_], [_] ,[] )  :-  !. 

/*  Is  this  an  edge  cell?  */ 

incrclumprow2 ( [Rll I RL1 ] ,_,  [_, G12 I GL1 ]  ,  [G21 , *  I GL2 ]  ,  [* I RL2 ]  )  :-  ! , 

incrclumprow2 (RL1, *, [G12 IGL1] , [* I GL2 ] ,RL2) . 
/*  Start  a  new  region  if  this  cell  is  unconnected  above  and  to  left  */ 
incrclumprow2 ( [Rll IRL1] ,_,[_,* IGL1]  ,  [*,G22 IGL2]  ,  [KIRL2]  )  :-  !,  rowegensym  (K ) , 

incrclumprow2 (RL1 ,  K, [ * IGL1] , [G2  2 I GL2 ] , RL2 ) . 
/*  Merge  with  region  above  if  this  cell  is  unconnected  to  left  */ 
incrclumprow2 ( [Rll IRL1] ,_,  [_,G12 IGL1]  ,  [*,G22|GL2]  ,  [Rll I RL2 ] )  : -  ! , 

incrclumprow2 (RL1,R11, [G12 IGL1] , [G22 IGL2] , RL2 ) . 
/*  Merge  with  region  left  if  this  cell  is  unconnected  above  */ 
incrclumprow2 ( [Rll IRL1] ,R20, [_, * IGL1] , [G21,G22 I GL2 ] , [R20IRL2] )  :-  ! , 

incrclumprow2 (RL1,R20, [* IGL1] , [G22 I GL2 ] , RL2 ) . 
/*  Else  merge  two  regions  through  this  cell;  lower  region  number  wins  */ 
incrclumprow2 ( [R11IRL1] ,R20, [_,G12 IGL1] , [G21,G22 IGL2] , [RAIRL2] )  :- 

not (R20  =  R11)  ,  sort2 (R20 , Rll , RA, RB) ,  not (region_name_change (RB, RA) ) ,  !, 

assertz (region_name_change (RB, RA) ) , 

incrclumprow2 (RL1,RA, [G12 IGL1 ] , [G22 IGL2] , RL2 ) . 
/*  Avoid  merge  if  two  regions  meeting  are  same  or  are  marked  for  merge  */ 
incrclumprow2 ( [Rll I RL1 ] , R20 , [_, G12 I GL1 ] , [G21 , G22 IGL2] , [RAIRL2] )  :-  ! , 

sort 2 (R11,R20,RA,RB)  ,  incrclumprow2 (RL1,RA,  [G12 IGL1] ,  [G22 I GL2 ] , RL2 )  . 

/*  Region  "clumping"  using  the  gradient- threshold  array  as  input,  */ 

/*  with  merging  across  diagonals  also  permitted.  */ 

/*  This  version  more  appropriate  for  sound  processing.  */ 

diagincrclump (Inf ile, Outfile)  :-  abolish (region_name_change/2 ) , 

abolish (lastrowegensym/1) ,  abolish ( final_code/2 ) , 

see (Inf ile),  tell ( tempclumpf ile) , 

readlineclump (GL1 ) ,  asterisk_buf f er (GL1 ,XGL1) , 

length(GLl,N) ,  Np2  is  N+2,  constant_vector ( * , Np2 , CV) , 

diagincrclumprow (CV ,CV , XGL1 ) , 

see (tempclumpf ile) ,  tell (Outfile), 

abolish (lastrowegensym/1 ) ,  f ix_region_names,  seen,  told, 
diagincrclumprow (XRL1 ,XGL1,XGL2)  :-  at_end_of_f ile, 


83 


diagincrclumprow2 (XRL1, *,XGL1 ,XGL2,RL2) ,  write_line (RL2 ) ,  seen,  told,  !. 
diagincrclumprow (XRL1 , XGL1 , XGL2 )  : - 

diagincrclumprow2 (XRL1, * , XGL1 ,XGL2 , RL2 ) , 

write_line(RL2 ) ,  readli nee lump (GL3 ) , 

asterisk_buffer (RL2,XRL2) ,  asterisk_buf f er (GL3 , XGL3 )  , 

diagincrclumprow (XRL2 ,XGL2,XGL3 ) . 
/*  Puts  asterisks  at  front  and  rear  of  list  */ 
asterisk_buffer (L, [* IXL] )  :-  append (L, [*], XL) ,  !. 

/*  Handles  a  single  row  during  diagincremental  clumping.  */ 
/*  First  arg  is  region  labels  for  previous  row,  second  arg  is  region  */ 
/*  label  for  cell  to  the  left  of  the  current  cell,  third  arg  is  list  */ 
/*  of  Os  and  *s  (gradient  threshold  output)  for  row  above,  fourth  arg  */ 
/*  is  same  for  current  row,  and  fifth  arg  is  output,  the  new  region  labels. 

/*  At  end  of  row?  */ 

diagincrclumprow2 (_,_,  [_,_] ,  [_,_] ,  [  ]  )  :-  ! ■ 

/*  Is  this  an  edge  cell?  */ 

diagincrclumprow2 ( [_IRL1] ,_,  [_, G12 I GL1 ] ,  [_, * IGL2] , [* IRL2] )  :-  !  , 

diagincrclumprow2 (RL1 , * , [G12 I GL1 ] , [*IGL2] , RL2) . 

/*  Start  a  new  region  if  this  cell  is  unconnected  in  all  4  directions  */ 
diagincrclumprow2 ([*,*,*  IRL1]  ,  *,  [*IGL1] ,  [* I GL2 ] ,  [KIRL2] )  :- 

!,  rowegensym(K) ,  diagincrclumprow2 ( [ * , *  I RL1 ] , K, GL1 , GL2 , RL2 )  . 
/*  Merge  adjacent  regions  through  this  cell;  lowest  region  number  wins  */ 
diagincrclumprow2 ( [Rll , R12 , R13 I RL1 ] , R21 , [_IGL1] , [_IGL2] , [KIRL2] )  :- 

sorted_numbers_in( (Rll , R12 , R13 , R21 ] , [KIKL] ) , 

update_neighbor_regions (K,KL) , 

diagincrclumprow2 ( [R12, R13 I RL1 ] ,K,GL1 ,GL2 , RL2 ) . 
diagincrclumprow2 (RL1 ,R21 ,GL1 ,GL2 ,RL2 )  :-  told,  seen, 

write ( 'Cannot  clump  this  situation:'),  nl , 

write( (RL1,R21,GL1,GL2,RL2] ) ,  nl,  !,  foo, 

diagincrclumprow2 (RL1,R21 , GL1 ,GL2 , RL2 ) . 

/*  Returns  the  sorting  of  all  numbers  in  a  list  of  symbols  */ 
sorted_numbers_in (L, SL)  :-  setof (N, number_member (N, L) , NL) ,  sort(NL,SL),  !. 
number_member (N, L)  :-  append (_, [N I _], L) ,  number (N). 
update_neighbor_regions (K, [ ) ) . 
update_neighbor_regions (K, [K2 IKL] )  : - 

( region_name_change (K2 , K) ;  assertz ( region_name_change (K2 , K) ) )  ,  !  , 

update_neighbor_regions (K , KL) . 

/*  Function  that  returns  the  final  region  name  to  use  after  all  merging  */ 
transitive_region_name_change (R,R)  :-  not ( region_name_change (R, R2 ) ) ,  !. 
transitive_region_name_change (Rl, R2 )  :-  region_name_change (Rl, R3 ) ,  !, 

transitive_region_name_change2 (R3 , R2) . 
transitive_region_name_change2 (Rl , R2)  : -  region_name_change (Rl , R3 ) , 

transitive_region_name_change2 (R3 , R2 ) . 
transitive_region_name_change2 (R, R)  :-  !. 

/*  Modified  region  "clumping"  using  the  intensities  themselves;  */ 

/*  compares  differences  of  adjacent  cells  to  grow_threshold .  */ 

/*  Puts  -10000  boundary  around  the  whole  picture.  */ 

/*  This  version  more  appropriate  for  picture  brightness  processing.  */ 

incrxclump ( Inf ile, Outf ile)  :-  abolish ( region_name_change/2 ) , 

abolish (lastrowegensym/1 ) ,  abolish ( final_code/2 ) , 

see (Infile),  tell ( tempxclumpf ile) , 

readli nee lump (GL1 ) ,  length (GL1 , N) ,  constant_vector (- 10  000, N, CV) , 

incrxclumprow(CV,CV,GLl ) , 

see (tempxclumpf ile) ,  tell (Outf ile) , 

abolish (lastrowegensym/1 ) ,  f ix_region_names,  seen,  told, 
incrxclumprow (RL1 ,GL1 ,GL2 )  :- 

a  t_end_o  f _f  i 1 e , 

incrxclumprow2 (RL 1,-10000,  [-10000 IGL1 ] ,  [-10000  I GL2 ]  , RL2 ) ,  wri te_l ine (RL2 ) 

seen,  told,  ! . 
incrxclumprow (RL1 , GL1 , GL2 )  :- 

incrxclumprow2 (RL1, -10000,  [-10000 IGL1] ,  [-10000  I GL2 ] , RL2 ) , 

write_line(RL2) ,  readlineclump (GL3 ) ,  incrxclumprow (RL2 , GL2 , GL3 ) . 
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/*  Handles  a  single  row  during  incremental  x-clumping.  */ 

/*  First  arg  is  region  labels  for  previous  row,  second  arg  is  region  */ 

/*  label  for  cell  to  the  left  of  the  current  cell,  third  arg  is  list  */ 

/*  of  Os  and  *s  (gradient  threshold  output)  for  row  above,  fourth  arg  */ 

/*  is  same  for  current  row,  and  fifth  arg  is  output,  the  new  region  labels.  */ 

/*  At  end  of  row?  */ 

incrxclumprow2 ([],_,  [_]  ,  [_]  ,  ( ] )  :-  !  . 

/*  Start  a  new  region  if  this  cell  is  unconnected  above  and  to  left  */ 

incrxclumprow2 ( [Rll IRL1] ,_, [_,G12 IGL1] , [G21,G22 I GL2 ] , [KIRL2] )  :- 

grow_threshold(G) ,  deviation (G22 , G12 , DY) ,  DY>G,  deviation (G22 , G21 , DX) ,  DX>G, 

! ,  rowegensym(K) ,  incrxclumprow2 (RL1 ,  K, [G12 IGL1] , [G22 IGL2] , RL2 ) . 
/*  Merge  with  region  above  if  this  cell  is  unconnected  to  left  */ 
incrxclumprow2 ( (Rll IRL1] ,_,  [_,G12 IGL1] ,  [G21,G22 I GL2 ] ,  [Rll IRL2] )  :- 

grow_threshold(G) ,  deviation (G22 , G21 , DX) ,  DX>G,  !, 

incrxclumprow2 (RL1,R11, [G12 IGL1 ] , [G22 IGL2] , RL2 ) . 
/*  Merge  with  region  left  if  this  cell  is  unconnected  above  */ 
incrxclumprow2 ( (Rll IRL1] ,R20, [_,G12 IGL1] , [G21,G22 I GL2 ] , [R20 |RL2] )  :- 

grow_threshold(G) ,  deviation (G22 , G12 , DY) ,  DY>G,  !, 

incrxclumprow2 (RL1,R20, [G12 IGL1] , [G22 IGL2] , RL2 ) . 
/*  Else  merge  two  regions  through  this  cell;  lower  region  number  wins  */ 
incrxclumprow2 ( [Rll IRL1] ,R20, [_,G12 IGL1] , [G21,G22 IGL2] , [RAIRL2] )  :- 

not (R20=R11 ) ,  sort2 (R20,R11 ,RA, RB) ,  not ( region_name_change (RB, RA) ) ,  !, 

assertz (region_name_change (RB, RA) ) , 

incrxclumprow2 (RL1,RA, [G12 IGL1] , [G22IGL2] , RL2 ) . 
/*  Avoid  merge  if  two  regions  meeting  are  same  or  are  marked  for  merge  */ 
incrxclumprow2 ( [Rll IRL1] ,R20, [_,G12 IGL1] , [G21,G22 IGL2] , [RAIRL2] )  :-  ! , 

sort2 (R11,R20,RA,RB) ,  incrxclumprow2 (RL1 , RA, [G12 IGL1] , [G2  2 I GL2 ] , RL2 ) . 

/*  Computes  area,  circumference,  length  and  width  of  each  region  */ 
/*  in  an  array  of  region  codes.  */ 

incrshapes ( Inf ile, Outf ile)  :- 

abolish (area/2 ) ,  abolish (circumference/2 ) ,  abolish (xmax/2 ) , 

abolish (xmin/2 ) ,  abolish (ymax/2 ) ,  abolish (ymin/2 ) , 

abolish (width/2 ) ,  abolish (height/2 ) , 

see(Infile),  readlineclump (LI ) ,  readlineclump (L2 ) , 

length! LI ,N) ,  cons tant_vec tor (*,N,CV) , 

incrshapes2 (1 ,CV, LI , L2 ) ,  seen, 

tell (Outf ile) ,  listing (area) ,  listing (circumference) , 

f igure_dimensions,  listing (width) ,  listing (height ) ,  told. 
incrshapes2 (Row, LI , L2 , L3 )  :-  at_end_of_f ile,  !, 

length(L2,N) ,  constant_vector ( * , N, CV) , 

append (LI, [*] ,XL1) ,  append (L2 ,[*], XL2 ) ,  append (L3 ,[*], XL3 ) , 

incrshapes3 (Row, 1, [* |XL2] , [*IXL1] , [* IXL3] ) ,  Rowpl  is  Row+1, 

incrshapes3 (Rowpl, 1, [* IXL3] , [*IXL2] , [*, * ICV] ) ,  ! . 
incrshapes2 (Row,Ll, L2,L3)  :-  append (LI ,[*], XL1 ) ,  append (L2 ,(*], XL2 ) , 

append (L3, [*] , XL3 ) ,  incrshapes3 (Row,l, [* IXL2] , [* IXL1] , [* IXL3] ) , 

readlineclump (L4 ) ,  Rowpl  is  Row+1,  incrshapes2 (Rowpl , L2 , L3 , L4 )  . 
incrshapes 3 (Row, Col , [_,_] , [_,_] , [_,_] ) • 
incrshapes3 (Row, Col,  [121,  *  I  L2  ]  ,  [111, 112 IL1]  ,  [131,  132  I L3 ] )  :-  !, 

Colpl  isCol+1,  incrshapes3 (Row,Colpl, [* IL2) , [112 IL1 ] , [132 IL3) ) . 
incrshapes3 (Row, Col,  (I21,I22,I23IL2],  [111,1121  LI],  [ 131 , 132  I L3 ] )  :- 

uninteresting_region( 122 ) ,  !,  Colpl  is  Col+1, 

incrshapes3 (Row, Colpl,  [ 12  2 , 12  3  I L2 ] ,  [112  I  LI] ,  [132  I L3 ] )  . 
incrshapes3 (Row, Col, [I21,I22,I23IL2],[I11,I12IL1],[I31,I32IL3])  :-  !, 

update_area (122) ,  update_circumference (122, 121, 112, 123, 132), 

update_dimensions (122 , Row, Col ) ,  Colpl  is  Col+1,  !, 

incrshapes3 (Row, Colpl,  [I22,I23|L2],  [1121  LI],  [I32IL3])  . 

update_area (R)  :-  retract (area (R,N) ) ,  !,  Npl  is  N+l,  assertz (area (R, Npl )) ,  !. 
update_area (R)  :-  assertz (area (R, 1 )) ,  !. 

update_circumference(R, R, R, R,R)  :-  !. 

update_circumference (R, _,_,_,_)  :-  retract (circumference (R , N) ) ,  !, 
Npl  is  N+l,  assertz (circumference (R, Npl) ) ,  !. 
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update_circumf erence (R,  _,_,  _,_)     :-   assertz (circumference (R, 1 )) ,    !. 

update_dimensions (R, Row, Col )    :- 

xmax(R,XH),    Col=<XH,    xmin(R,XL),    Col>=XL, 

ymax(R,YH),  Row=<YH,  ymin (R, YL) ,  Row>=YL,  !. 
update_dirnensions (R, Row, Col )  :- 

retract (xmax(R,XH) ) ,  retract (xmin (R,XL) ) , 

retract (ymax(R,YH) ) ,  retract (ymin (R, YL) ) ,  !, 

max(Col,XH,NXH) ,  min (Col , XL, NXL) ,  max (Row, YH, NYH) ,  min  (Row, YL, NYL)  , 

assertz (xmax(R,NXH) ) ,  assertz (xmin (R, NXL) ) , 

assertz (ymax(R, NYH) ) ,  assertz (ymin (R, NYL) ) ,  !. 
update_dimensions (R, Row, Col )  :- 

assertz (xmax (R, Col ) ) ,  assertz (xmin (R, Col ) ) , 

assertz (ymax (R, Row) ) ,  assertz (ymin (R, Row) ) ,  !. 

f igure_dimensions  :-  xmax(R,XH),  xmin(R,XL),  DX  is  1+XH-XL, 

ymax(R,YH),  ymin(R,YL),  DY  is  1+YH-YL, 

assertz (width(R,DX) ) ,  assertz (height (R, DY) ) ,  fail, 
f igure_dimensions . 

/*  Converts  a  picture  file  to  pixel (X, Y, Value)  facts,  writes  them  out.  */ 
/*  All  cells  labelled  "*"  are  ignored.  */ 
pixel_convert ( Inf ile, Outf ile)  :-  abolish (pixel/3 ) , 

see(Infile),  pixel_convert2 ( 1 ) ,  seen, 

tell (Outf ile) ,  listing (pixel ) ,  told. 
pixel_convert2 (Row)  :-  not (at_end_of_f ile) ,  readlineclump (L) , 

pixel_convert3 ( 1 , Row, L) ,  Rowpl  is  Row+1,  !,  pixel_convert2 (Rowpl ) ,  !. 
pixel_convert2 (Row) . 
pixel_convert3 (Col , Row,  ( I  I L] )  : -  {!='*';    assertz (pixel (Col , Row, I )  ) )  , 

Colpl  is  Col+1,  !,  pixel_convert3 (Colpl , Row, L) . 
pixel_convert3 (Col , Row, [ ] ) . 

/*  Reads  a  file  and  correlates  its  entries  with  current  pixel (X, Y, Value)  */ 
/*  facts,  makes  the  value  at  the  corresponding  place  in  the  input  file  */ 
/*  to  be  the  fourth  argument  to  a  "pixel"  which  it  writes  out.  */ 
pixel_join  (Inf  ile,  Outf  ile)  -.- 

see(Infile),  tell (Outf ile) ,  pixel_join2 (1 ) ,  seen,  told. 
pixel_join2 (Row)  :-  not (at_end_of_f ile) ,  readlineclump (L) , 

pixel_join3 ( 1 , Row, L) ,  Rowpl  is  Row+1,  !,  pixel_join2 (Rowpl ) ,  !. 
pixel_join2 (Row) . 
pixel_join3 (Col ,Row,  [ I  I L] )  :-  pixel (Col , Row, R) ,  !, 

writeq(pixel (Col , Row, R, I )  ) ,  writef'.'),  nl , 

Colpl  is  Col+1,  !,  pixel_join3 (Colpl, Row, L) . 
pixel_join3 (Col ,Row,  ( I  I L]  )  :-  Colpl  is  Col  +  1,  !,  pixel_join3 (Colpl , Row, L)  . 
pixel_join3 (Col , Row, [ ] ) . 

/*  Converts  a  picture  file  to  CLIPS  form  and  writes  out  a  new  file  */ 
clips_convert ( Inf ile, Outf ile)  :-  see(Infile),  tell (Outf ile) , 

write (' (def facts  incrvision  '),  nl ,  clips_convert2 ( 1 ) , 

write (')') /  seen,  told. 
clips_convert2 (Row)  :-  not (at_end_of_f ile) ,  readlineclump (L) , 

clips_convert3 (1, Row, L) ,  Rowpl  is  Row+1,  clips_convert2 (Rowpl ) . 
clips_convert2 (Row) . 

clips_convert3 (Col, Row,  [* |L] )  :-  !, Colpl  is  Col  +  1,  clips_convert3 (Colpl , Row, L) 
clips_convert3 (Col , Row,  [ I  I L] )  :-  write ('( region_index  '),  write(Col), 

writer  '),  write(Row),  writer  '),  write(I),  write(')'),  nl , 

Colpl  is  Col+1,  clips_convert3 (Colpl , Row, L) . 
clips_convert3 (Col , Row, [ ] ) . 

/*  Utility  functions  */ 

/*  Some  input  routines.  */ 

/*  readlineclump  (<list>) --reads  a  line  from  the  terminal  and  */ 

/*  clumps  it  into  words,  then  binds  its  argument  to  this  list  */ 

/*  readline(<list>) --reads  a  line  from  the  terminal  and  */ 

/*  binds  its  argument  to  this  as  a  character  string  */ 

/*  niceread(<list>) --reads  a  line  from  the  terminal  and  */ 

/*  binds  its  argument  to  its  ascii  list  representation  */ 
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/*  Note:  This  assumes  that  spaces,  tabs,  commas,  colons,  and 
/*  semicolons  separate  words.  If  other  symbols  do  too,  */ 
/*  add  more  "terminator"  defintions.  */ 


readli nee  lump (L) 


niceread(S),  clumpstring (S, AL) ,  makenames (AL, L) , 


makenames  (  [  ] , 
makenames ( [ AL 


LL] , [nlinll; 


-  name (NL, AL) ,  makenames (LL, NLL) 


clumpstring(L3, [LI IL5] )  :-  nextclump (L3 , [ ] , LI , L2 ) ,  ! 
clumpstring (L, [L] )  :-  member (X, L) ,  not ( terminator (X) 
clumpstring (L,  [ ] )  . 
nextclump ( [ ] , LI 


clumpstring (L2 , L5 ) 


RL1,[])   :- 

not (Ll= [ ] ) ,  ! ,  reve 

[  ] ,L2,L3) 

:-  terminator (T) ,  !, 

L1,RL1,L) 

:-  terminator (T) ,  !, 

L1,L2,L3) 

:-  nextclump (L,  [X 1  LI 

nextclump (L, ( ] 
reverse (LI , RL1 
] ,L2,L3) . 


nextclump ( [TIL] 
nextclump ([TIL] 
nextclump ( [X  I L] 
terminator) 9) . 
terminator ( 32 )  . 
terminator (44 )  . 
terminator ( 58  )  . 
terminator ( 59 ) . 

readline(S)  :-  niceread(L),  name(S,L). 
niceread(U)  :-  at_end_of_f ile,  !. 
niceread(M)  :-  at_end_of_line,  getO  (_)  ,  !. 
niceread( [ACI AL] )  :-  getO(AC),  niceread (AL) . 

constant_vector (Symbol , 0, [] )  :-  !. 
constant_vector (Symbol , Length,  [Symbol  I  List ] )  :- 
Lengthml  is  Length-1,  constant_vector (Symbol , Lengthml , List ) 


L2,L3) 


sort2 (N1,N2,N1,N2) 
sort2 (N1,N2,N2,N1) 


-  N1<N2, 
i 


/*  Generates  a  new  integer  each  time  it  is  called 
rowegensym(K)  :-  retract (lastrowegensym (Kml )) ,  !, 
asserta ( lastrowegensym (K) ) . 
rowegensym ( 1 )  :-  asserta (lastrowegensym ( 1 )) . 


is  Kml+1 


87 


APPENDIX  D:  CLOS  CODE  FOR  THE  CORRELATION 

PROGRAM 


(defconstant  *region-match-criterion*  0.4) 

(defmethod  rb  ( ) 
require  :process) 

let  (  (blackboardl  (rnake-instance  'blackboard)) 
blackboard2  (make-instance  'blackboard)) 
picture  (make- instance  'screen))) 

initialize-blackboard  blackboardl  'bonele  'bone2e  0  3  'lower) 
init ialize-blackboard  blackboard2  'bone3e  'bone4e  181  184  'upper) 
start-picture  picture) 
run-blackboard-r  blackboardl  picture) 
run-blackboard-r  blackboard2  picture) 
match- far-phones  blackboardl  blackboard2  picture) 
) 

(defmethod  run-blackboard-r  ( ( the-blackboard  blackboard)  (picture  screen)) 
with-slots  (phonel  phone2 )  the-blackboard 
let  ( (new- region  (gentemp))  listen-status) 
set  new-region  (make-instance  'region)) 
if  (<  (phone-time  phonel)  (phone-time  phone2 ) ) 
let  () 

setf  listen-status  (listen-to-hydrophone  new-region  phonel)) 
unless  listen-status 

setf  listen-status  (listen-to-hydrophone  new-region  phone2 ) ) ) ) 
let  () 

setf  listen-status  (listen-to-hydrophone  new-region  phone2 ) ) 
unless  listen-status 

setf  listen-status  (listen-to-hydrophone  new- region  phonel))))) 
when  listen-status 

process-new- region  new-region  the-blackboard  picture) 
if  (>  (-  (possible-region-match-time  the-blackboard) 
last-definite-region-match- time  the-blackboard) ) 
*  6  (allowable- time-difference  the-blackboard))) 
match-definite-regions  the-blackboard  picture)) 
run-blackboard-r  the-blackboard  picture) ) ) ) ) 

(defmethod  initialize-blackboard  ((the-blackboard  blackboard)  phonel- filename 
phone2- filename  phonel -latitude  phone2- latitude  screen-position) 
(with-slots  (phonel  phone2 )  the-blackboard 

(start-hydrophone  phonel  phonel- filename  phonel-latitude  screen-position) 
(start-hydrophone  phone2  phone2- filename  phone2-latitude  screen-position) 
(setf  (screen-position  the-blackboard)  screen-position) 
(setf  (allowable- time-difference  the-blackboard) 

(*  (/  (abs  (-  (latitude  phonel)  ( latitude  phone2 )) )  0.81)  1.5)))) 
; Speed  of  sound  (SOS)  through  water  =0.81  nm/sec 
;Slop  factor  for  variation  of  SOS  and  phone  separation  is  1.5 

(defmethod  process-new-region 
(new-region  (the-blackboard  blackboard)  (picture  screen)) 
(with-slots  (phonel  phone2  unmatched-regions)  the-blackboard 
(if  (equal  (phone-id  (eval  new- region))  (phone-id  phonel)) 
(setf  (phone-time  phonel)  (time-min  (eval  new-region))) 
(setf  (phone-time  phone2 )  (time-min  (eval  new-region)))) 
(change-unmatched-to-unmatchable  new-region  the-blackboard  picture) 
(match-new- region-to-unmatched-regions  new-region  the-blackboard  picture) 
(match- new- region -to- tentative-matched- regions  new- region  the-blackboard 
picture) 

(unless  (member-p  new-region  (possible-matched-regions  the-blackboard) ) 
(setf  unmatched-regions  (append  unmatched-regions  (list  new-region))) 
(draw-region  new-region  the-blackboard  picture  1)))) 
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(defmethod  change- unmatched- to-unmatchable 
(new-region  (the-blackboard  blackboard)  (picture  screen)) 
(let  ( (unmatched- region-list  (unmatched- regions  the-blackboard))) 
(setf  (unmatched-regions  the-blackboard) 
( change -unmatched-to-unmatchable-r  unmatched- region- list 
new-region  the-blackboard  picture) ) ) ) 

(defmethod  change-unmatched- to-unmatchable-r  (unmatched- region-list  new-region 
(the-blackboard  blackboard)  (picture  screen)) 
(if  unmatched-region-list 
(let  ((tail  (rest  unmatched-region-list)) 
(head  (first  unmatched- region-list ))  ) 

(if  (or  (equal  (phone-id  (eval  new-region))  (phone-id  (eval  head))) 
(<=  (-  (time-min  (eval  new- region ) )  (time-max  (eval  head))) 
(allowable-time-difference  the-blackboard) ) ) 
(cons  head 

(change-unmatched-to-unmatchable-r  tail  new-region  the-blackboard 
picture) ) 
(let  () 

(setf  (definite-unmatched- regions  the-blackboard) 
(append  (definite-unmatched- regions  the-blackboard) 
(list  head) ) ) 

(draw-region  head  the-blackboard  picture  2) 
(change-unmatched-to-unmatchable-r  tail  new-region 
the-blackboard  picture) ) ) ) ) ) 

(defmethod  match-new- region- to -unmatched- regions 
(new-region  (the-blackboard  blackboard)  (picture  screen)) 
(let  ( (unmatched- region-list  (unmatched- regions  the-blackboard))) 
(setf  (unmatched- regions  the-blackboard) 

(match- new- regi on -to-unmatched-regions-r  unmatched- region- list 
new-region  the-blackboard  picture) ) ) ) 

(defmethod  match-new- region- to-unmatched-regions-r 
(unmatched- region-list  new-region  (the-blackboard  blackboard)  (picture  screen) 
(if  unmatched-region-list 
(let  ((tail  (rest  unmatched- region-list) ) 
(head  (first  unmatched-region-list))) 

(if  (equal  (phone-id  (eval  new-region))  (phone-id  (eval  head))) 
(cons  head  (match-new- region-to-unmatched-regions-r  tail  new-region 
the-blackboard  picture) ) 

(let*  ((region-match  (match-region-factor  new-region  head 
(allowable-time-difference  the-blackboard) ) ) 
( region-matching- f actor  (first  region-match))) 
(if  (<  region-matching-f actor  *region-match-criterion* ) 
(cons  head  (match-new- region-to-unmatched-regions-r  tail  new-region 
the-blackboard  picture) ) 
(let  () 

(adjust-blackboard- for-match  new-region  head  region-match 
the-blackboard  picture) 

(match-new-region-to-unmatched- regions-r  tail  new-region 
the-blackboard  picture) ))))))) 

(defmethod  adjust -blackboard- for -match 
(new-region  head  region-match  (the-blackboard  blackboard)  (picture  screen)) 
(let  ((region-match-factor  (first  region-match)) 
(match-dir  (second  region-match) ) ) 
(unless  (member  (region-id  (eval  new-region)) 
(possible-matched-regions  the-blackboard) ) 
(setf  (possible-matched-regions  the-blackboard) 

(append  (possible-matched-regions  the-blackboard)  (list  new-region))) 
(draw-region  new-region  the-blackboard  picture  3)) 
(setf  (possible-matched-regions  the-blackboard) 
(append  (possible-matched-regions  the-blackboard)  (list  head))) 
(let  ((dir  (direction  new-region  head  match-dir  (phonel  the-blackboard)))) 
(setf  (possible-region-matches  the-blackboard) 
(append  (possible-region-matches  the-blackboard) 
(list  (list  new-region  head  region-match-factor  dir))))) 
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(draw-region  head  the-blackboard  picture  3) 

(connect-matched- region  (list  new-region  head  region-match- factor ) 

picture  (phone-id  (eval  new- region) ) ) 

(setf  (possible-region-match- time  the-blackboard) 

(time-min  (eval  new-region))))) 

(defmethod  match-new- region- to- tentative-matched- regions 
(new-region  (the-blackboard  blackboard)  (picture  screen)) 
(let  ((matched-region-list  (possible-matched-regions  the-blackboard))) 
(setf  (possible-matched-regions  the-blackboard) 

(match-new- region- to- tentative-matched- regions- r  matched- region- list 
new-region  the-blackboard  picture)))) 

(defmethod  ma tch-new- region- to- tentati ve-matched- regions- r 
(matched-region-list  new-region  (the-blackboard  blackboard)  (picture  screen) 
(if  matched-region-list 

(let  ((tail  (rest  matched- region-list ) )  (head  (first  matched- region-1 ist )) ) 
(if  (or  (equal  (phone-id  (eval  new-region))  (phone-id  (eval  head))) 
(equal  new-region  head) 
(matched-member  (list  new- region  head) 
(possible-region-matches  the-blackboard) ) ) 

(cons  head  (match-new- region-to-tentative-matched- regions-r  tail  new-region 
the-blackboard  picture) ) 

(let*  ((region-match  (match-region-factor  new-region  head 
(allowable-time-difference  the-blackboard) ) ) 
( region-matching-factor  (first  region-match))) 
(if  (<  region-matching-factor  *region-match-criterion* ) 
(cons  head  (match-new- region-to- tentative-matched- regions-r  tail 
new-region  the-blackboard  picture) ) 
(let  () 

(unless  (member  (region-id  (eval  new-region)) 
(possible-matched-regions  the-blackboard) ) 
(setf  tail  (append  tail  (list  new-region)))) 
(let  ((dir  (direction  new-region  head 
(second  region-match)  (phonel  the-blackboard)))) 
(setf  (possible-region-matches  the-blackboard) 
(append  (possible-region-matches  the-blackboard) 
(list  (list  new-region  head  region-matching-factor 
dir) ) ) ) ) 

(draw-region  new- region  the-blackboard  picture  3) 
( connect-matched- region 

(list  new-region  head  region-matching-factor) 
picture  (phone-id  (eval  new- region) ) ) 
(setf  (possible-region-match- time  the-blackboard) 
(time-min  (eval  new- region) ) ) 

(cons  head  (match-new- region-to-tentative-matched- regions-r 
tail  new-region 
the-blackboard  picture) )))))))) 

(defmethod  match-region- factor  (regionl  region2  allowable-time-difference) 
(let*  ( (blobl  (eval  regionl))  (blob2  (eval  region2))) 
(if  (<  allowable-time-difference 
(abs  (-  (time-min  blobl)  (time-min  blob2)))) 
'  (0  0) 
(overlap-factor  blobl  blob2)))) 

(defmethod  matched-member  (two-regions  matched- list ) 
(if  matched-list 

(let  ((head  (first  matched-list ) )  (tail  (rest  matched-list )) ) 
(if  (and  (equal  (first  two-regions)  (first  head)) 
(equal  (second  two-regions)  (second  head))) 
t 
(matched-member  two-regions  tail))))) 

(defmethod  direction  (regionl  region2  offset  (phonel  hydrophone)) 
(let  ((tminl  (time-min  (eval  regionl)))  (tmin2  (time-min  (eval  region2)))) 
(if  (equal  (phone-id  (eval  regionl))  (phone-id  phonel) ) 
(if  (<  tminl  tmin2) 
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offset 

(-  offset)  ) 

(if  (<  tminl  tmin2) 

(-  offset) 

offset)  )  )  ) 

(defmethod  area-factor 
( (blobl  region)  (blob2  region)  ( the-blackboard  blackboard)) 
(let  (factor) 

(if  (<  (area  blobl)  (area  blob2)) 
(setf  factor  (/  (area  blobl)  (area  blob2))) 
(setf  factor  (/  (area  blob2 )  (area  blobl)))) 

(let  ((time-difference  (abs  (-  (time-min  blobl)  (time-min  blob2))))) 
(if  (>  time-difference  (*  2  (allowable-time-difference  the-blackboard))) 
(setf  factor  0) 

(setf  factor  (/  factor  (-  1  (/  (*  0.1  time-difference) 
(allowable-time-difference  the-blackboard) )))))) 
(if  (>  factor  1) 
(setf  factor  1 ) 
factor) ) ) 

(defmethod  overlap-factor 
( (blobl  region)  (blob2  region) ) 

(let  ((overlap-value  0)  overlap-index  temp-overlap-value) 
(do  ((index  (+  2  (abs  (-  (time-min  blobl)  (time-min  blob2))))  (1-  index))) 
( (-  index  (  -  (abs  (-  (time-min  blobl)  (time-min  blob2)))  3)) 
overlap-value) 

(if  (<  (time-min  blobl)  (time-min  blob2)) 

(setf  temp-overlap-value  (overlap-region  index  blobl  blob2)) 
(setf  temp-overlap-value  (overlap-region  index  blob2  blobl))) 
(when  (>  temp-overlap-value  overlap-value) 
(setf  overlap-value  temp-overlap-value) 
(setf  overlap-index  index) ) ) 
(if  overlap-index 

(if  (<  (time-min  blobl)  (time-min  blob2)) 
(list  overlap-value  overlap-index) 
(list  overlap-value  (-  overlap-index))) 
'  (0  0)  )  )  ) 

(defmethod  overlap-region  (time-diff  (blobl  region)  (blob2  region)) 
(let  (time-adjusted-region) 

(setf  time-adjusted-region  (time-adjust  time-diff  (pixels  blobl))) 
(let*  ((overlap-area  (overlap-pixels  time-adjusted-region  (pixels  blob2 ) ) ) 
(/  (*  2  overlap-area) 
(+  (length  (pixels  blobl))  (length  (pixels  blob2))))))) 

(defmethod  time-adjust  (time-diff  pixels) 
(mapcar  #' (lambda  (x)  (list  (+  (first  x)  time-diff)  (second  x) ) )  pixels)) 

(defmethod  overlap-pixels  (pixel-setl  pixel-set2) 
(if  pixel-setl 

(if  (pixel-member-p  (first  pixel-setl)  pixel-set2) 
(append  (list  (first  pixel-setl)) 
(overlap-pixels  (rest  pixel-setl) 
(remove  (first  pixel-setl)  pixel-set2 ) ) ) 
(overlap-pixels  (rest  pixel-setl)  pixel-set2 ) ) ) ) 

(defmethod  overlap-pixels  (regionl  region2) 
(let  ((overlap-value  0)) 

(if  (<  (length  regionl)  (length  region2)) 
(dolist  (pixel  regionl  overlap-value) 
(if  (pixel-member-p  pixel  region2) 
(setf  overlap-value  (1+  overlap-value) ) ) ) 
(dolist  (pixel  region2  overlap-value) 
(if  (pixel-member-p  pixel  regionl) 
(setf  overlap-value  (1+  overlap-value))))))) 

(defmethod  pixel-member-p  (item-list  list -of  -  item-lists) 
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(if  list-of-item-lists 

(let  ((head  (first  list-of-item-lists))  (tail  (rest  list-of-item-lists))) 

(if  (and  (=  (first  item-list)  (first  head)) 

(=  (second  item-list)  (second  head))) 

t 

(pixel-member-p  item-list  tail))))) 

(defmethod  min-freq- factor  ( (blobl  region)  (blob2  region)) 
( let  ( ( freq-f actor 

(-  1  (/  (abs  (-  (freq-min  blob2)  (freq-min  blobl)))  64)))) 
(*  freq-factor  f req-factor ) ) ) 
; There  are  64  Hz  in  the  frequency  band. 

(defmethod  match-definite-regions  ((the-blackboard  blackboard)  (picture  screen)) 
(let  ((head  (first  (possible-region-matches  the-blackboard))) 
(tail  (rest  (possible-region-matches  the-blackboard)))) 
(if  head 
(match-def inite-regions-r  head  tail  the-blackboard  picture) ) ) ) 

(defmethod  match-def inite-regions-r 
(possible -match  possible-matched- list 
(the-blackboard  blackboard)  (picture  screen)) 
(when  possible-matched-list 

(if  (check-time-f rame-r  possible-match  the-blackboard) 
(let  ((multiple-matches  (f ind-multiple-matches-r  possible-match 
possible-matched- list 
the-blackboard) ) ) 
(if  multiple-matches 

(perf orm-optimization-f or-multiple-matches) 
(change-possible-match- to-definite  possible -match 
the-blackboard  picture) ) 

(match-def inite-regions-r  (first  possible-matched-list) 
(rest  possible-matched-list) 
the-blackboard  picture) ) ) ) ) 

(defmethod  change- possible-match- to-definite 
possible-match  (the-blackboard  blackboard)  (picture  screen)) 
setf  (possible-region-matches  the-blackboard) 

remove  possible-match  (possible-region-matches  the-blackboard) ) ) 
setf  (definite-region-matches  the-blackboard) 

append  (list  possible-match)  (definite-region-matches  the-blackboard))) 
setf  (possible-matched-regions  the-blackboard) 
remove  (first  possible-match) 
possible-matched-regions  the-blackboard) )) 
setf  (possible-matched-regions  the-blackboard) 
remove  (second  possible-match) 
possible-matched-regions  the-blackboard) ) ) 
setf  (definite-matched- regions  the-blackboard) 
append  (definite-matched- regions  the-blackboard) 
list  (first  possible-match)))) 
setf  (definite-matched-regions  the-blackboard) 
append  (definite-matched- regions  the-blackboard) 
list  (second  possible-match)))) 

setf  (last-definite-region-match-time  the-blackboard) 
time-min  (eval  (first  possible-match)))) 

draw-region  (second  possible-match)  the-blackboard  picture  4) 
draw-region  (first  possible-match)  the-blackboard  picture  4)) 

defmethod  f ind-mul ti pie-matches- r 

matched-region  region-matches  (the-blackboard  blackboard)) 

let  ((head  (first  region-matches))) 

if  region-matches 

if  (region-in-common  matched- region  head) 

when  t 

check-time-f rame-r  head  the-blackboard) 

cons  head 

f ind-multiple-matches-r  matched-region  (rest  region-matches) 
the-blackboard) ) ) ) 
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( f ind-multiple-matches-r  matched-region  (rest  region-matches) 
the-blackboard) ) ) ) 

(defmethod  region-in-common  (matched-regionl  matched-region2 ) 
(if  (member  (first  matched-regionl)  matched-region2 ) 
(first  matched-regionl) 

(if  (member  (second  matched-regionl )  matched-region2) 
(second  matched-region2 ) ) ) ) 

(defmethod  perform-optimization-f or-mul tiple-matches  () 
(print  "Perf orm-optimization- for-multiple-matches  is  a  stub")) 

(defmethod  check-time-f rame-r  (possible-matched- region 
(the-blackboard  blackboard)) 

(if  (<  (time-max  (eval  (first  possible-matched-region))) 
(-  (possible-region-match-time  the-blackboard) 
(*  3  (allowable-time-difference  the-blackboard)))) 
(if  (<  (time-max  (eval  (second  possible-matched-region))) 
(-  (possible-region-match-time  the-blackboard) 
(*  3  (allowable-time-difference  the-blackboard)))) 
t))  ) 


(defclass  blackboard  () 
( (phonel  : accessor  phonel 

initform  (make-instance  'hydrophone)) 
;phone2  : accessor  phone2 

initform  (make-instance  'hydrophone)) 
[screen-position  :accessor  screen-position 

initform  'lower) 
(unmatched- regions  raccessor  unmatched-regions 

initform  nil ) 
(possible-matched-regions  : accessor  possible-matched-regions 

initform  nil ) 
(possible-region-matches  :accessor  possible-region-matches 

initform  nil ) 
[possible -region-match -time  : accessor  possible -region -match -time 

initform  0) 
[definite-matched-regions  :accessor  definite-matched- regions 

initform  nil ) 
(definite-region-matches  :accessor  definite- region-matches 

initform  nil ) 

last-definite- region-match- time  : accessor  last-definite- region -match -time 

initform  0) 
[definite-unmatched-regions  raccessor  definite-unmatched- regions 

initform  nil) 

(allowable-time-difference  :accessor  allowable-time-difference) ) ) 
[defclass  blackboard-object  ()  ()  ) 

[defclass  hydrophone  () 
((file  :initarg  :file 
:accessor  file) 

(blackboard-id  :initarg  'lower 
:accessor  blackboard-id) 
(phone-id  :initarg  :phone-id 
raccessor  phone-id) 
(latitude  :initarg  : latitude 
raccessor  latitude) 
(phone-time  : initform  0 
raccessor  phone-time) ) ) 


[defclass  region  () 
((blackboard-id  rinitarg  'lower 
raccessor  blackboard-id) 
(phone-id  rinitarg  rphone-id 
•.accessor  phone-id) 
(region-id  rinitarg  :region-id 
raccessor  region-id) 
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(region-type  accessor  region-type) 

(pixels  :initarg  :pixels 

:accessor  pixels) 

(time-min  :initarg  : time-min 

:accessor  time-min) 

(time-max  :initarg  :time-max 

: accessor  time-max) 

(freq-min  :initarg  :freq-min 

: accessor  freq-min) 

(freq-max  :initarg  :freq-max 

:accessor  freq-max) 

(area  :initarg  :area 

: accessor  area) ) ) 

(defclass  unmatched- regions  (region) 
((region-list  :initformnil 
accessor  region-list))) 

(defmethod  start-hydrophone  ( (phone  hydrophone)  filename  input-latitude 
screen-position) 

(with-slots  (file  latitude  phone-id  blackboard-id)  phone 
{setf  file  filename) 
(setf  latitude  input-latitude) 
(setf  phone-id  (gentemp) ) 
(setf  blackboard-id  screen-position) 
(setf  file  (open  filename  :direction  :input)))) 

(defmethod  listen-to-hydrophone  (region-name  (phone  hydrophone)) 
(let  ((blob  (eval  region-name))) 

(setf  (phone-id  blob)  (phone-id  phone)) 

(when  (setf  (pixels  blob)  (read  (file  phone)  nil  nil)) 

(setf  (blackboard-id  blob)  (blackboard-id  phone) ) 

(setf  (region-id  blob)  region-name) 

(setf  (region-type  blob)  (first  (pixels  blob))) 

(setf  (pixels  blob)  (rest  (pixels  blob))) 

(setf  (time-min  blob)  (get-min-time  (pixels  blob))) 

(setf  (time-max  blob)  (get-max-time  (pixels  blob))) 

(setf  (freq-min  blob)  (get-min-f req  (pixels  blob))) 

(setf  (freq-max  blob)  (get-max-f req  (pixels  blob))) 

(setf  (area  blob)  (length  (pixels  blob)))))) 

(defmethod  get-min-time  (pixel-list) 
(get-min- time-r  (first  (first  pixel-list))  (rest  pixel-list))) 

(defmethod  get-min-time-r  (temp-min  pixel-list) 
(if  (first  pixel-list) 

(let  ( (next -time  (first  (first  pixel-list)))) 
(if  (<  temp-min  next-time) 

(get-min-time-r  temp-min  (rest  pixel-list)) 
(get-min-time-r  next-time  (rest  pixel-list)))) 
temp-min) ) 

(defmethod  get-max-time  (pixel-list) 
(get-max-time-r  (first  (first  pixel-list))  (rest  pixel-list))) 

(defmethod  get-max-time-r  (temp-max  pixel-list) 
(if  (first  pixel-list) 

(let  ( (next -time  (first  (first  pixel- list ))) ) 
(if  (>  temp-max  next-time) 

(get-max-time-r  temp-max  (rest  pixel-list)) 
(get-max-time-r  next-time  (rest  pixel-list)))) 
temp-max) ) 

(defmethod  get-min-freq  (pixel-list) 
(get-min-f req-r  (second  (first  pixel-list))  (rest  pixel-list))) 

(defmethod  get-min-f req-r  (temp-min  pixel-list) 
(if  (first  pixel-list) 
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(let  ( (next-f req  (second  (first  pixel-list)))) 
(if  (<  temp-min  next-f req) 

(get-min-f req-r  temp-min  (rest  pixel-list)) 
(get-min-f req-r  next-freq  (rest  pixel-list)))) 
temp-min) ) 

(defmethod  get-max-freq  (pixel-list) 
(get-max-f req-r  (second  (first  pixel-list))  (rest  pixel-list; 

(defmethod  get-max-f req-r  (temp-max  pixel-list) 
(if  (first  pixel-list) 

(let  ((next-freq  (second  (first  pixel -1 ist ))) ) 
(if  (>  temp-max  next-freq) 

(get-max- f req-r  temp-max  (rest  pixel-list)) 
(get-max-f req-r  next-freq  (rest  pixel-list)))) 
temp-max) ) 


(defconstant  *pixel-size*  1) 
(defvar  *offset*  (*  *pixel-size*  64)) 
(defconstant  *low*  60) 
(defconstant  *high*  350) 

(defclass  screen  () 
(  ( id  accessor  id) 
(borders  :initform  5 
: accessor  borders) 
( left  : init f orm  0 
:accessor  left) 
(bottom  :initform  0 
: accessor  bottom) 
(width  :initform  1150 
accessor  width) 
(height  :initform  700 
accessor  height) 

(title  :initform  "Hydrophone  Region  Matches" 
:accessor  title) 
(activate-p  :initform  t 
:accessor  activate-p) 

( lower-y- text-position  :accessor  lower-y-text-position 
: initform  1000) 

(upper-y- text-position  :initform  1000 
accessor  upper-y-text-position) ) ) 

(defmethod  start-picture  ( (picture  screen) ) 
(require  :xcw) 

(cw: initial ize- common -windows) 

(setf  (id  picture)  (cw :make-window-stream  :borders  (borders  picture) 
:left  (left  picture) 
:bottom  (bottom  picture) 
:width  (width  picture) 
:height  (height  picture) 
:title  (title  picture) 
:activate-p  (activate-p  picture))) 

(setf  (cw:window-stream-extent-width  (id  picture))  (*  2  (width  picture))) 
(cw : enable-window-stream-extent-scrolling  (id  picture)  :vertical  nil 
:horizontal  t) 
(draw-y-scale  picture) 
(draw-time-line  picture) ) 

(defmethod  draw-y-scale  ( (picture  screen) ) 
(cw:draw-line-xy  (id  picture)  0  *low*  (*  2  (width  picture))  *low*) 
(cw:draw-line-xy  (id  picture)  0  (+  *low*  (*  *pixel-size*  64)) 
(*  2  (width  picture) )  (+  *low*  (*  *pixel-size*  64))) 
(cw:draw-line-xy  (id  picture)  0  (+  *low*  (*  *pixel-size*  128)) 
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(*  2  (width  picture))  (+  *low*  (*  *pixel-size*  128))) 
(cw:draw-line-xy  (id  picture)  0  *high*  (*  2  (width  picture))  *high* ) 
(cw:draw-line-xy  (id  picture)  0  (+  *high*  (*  *pixel-size*  64)) 
(*  2  (width  picture))  (+  *high*  (*  *pixel-size*  64))) 
(cw:draw-line-xy  (id  picture)  0  (+  *high*  (*  *pixel-size*  128)) 
(*  2  (width  picture) )  (+  *high*  (*  *pixel-size*  128)))) 

(defmethod  draw- time-1 ine  ((picture  screen)) 
(do  ((time-step  0  (  +  (*  *pixel-size*  60)  time-step))) 
( (>  time-step  (*  2  (width  picture)))  nil) 

(cw:draw-line-xy  (id  picture)  time-step  *high*  time-step  (-  *high*  3)) 
(cw:draw-line-xy  (id  picture)  time-step  *low*  time-step  (-  *low*  2)))) 

(defmethod  paint-picture  ( (picture  screen)  (the-blackboard  blackboard) ) 

(draw-regions-r  (definite-unmatched-regions  the-blackboard)  the-blackboard 
picture  2) 

(draw-regions-r  (possible-matched-regions  the-blackboard)  the-blackboard 
picture  3) 

(draw-regions-r  (unmatched-regions  the-blackboard)  the-blackboard 
picture  1) 

(connect -ma tched- regions- r  (possible- region -matches  the-blackboard) 
picture  (phone-id  (phonel  the-blackboard) ) ) ) 

(defmethod  draw-regions-r 
(region-list  (the-blackboard  blackboard)  (picture  screen)  shade) 
(when  region-list 

(draw-region  (first  region-list)  the-blackboard  picture  shade) 
(draw-regions-r  (rest  region-list)  the-blackboard  picture  shade))) 

(defmethod  draw-region 
(region  (the-blackboard  blackboard)  (picture  screen)  shade) 
(with-slots  (phonel  phone2 )  the-blackboard 

(let*  ((blob  (eval  region))  (screen-position  (blackboard-id  (eval  blob))) 
offset) 

(if  (equal  screen-position  'lower) 
(setf  offset  *low*) 
(setf  offset  *high*) ) 

( identi fy-region  (id  picture)  region  offset 
(phone-id  blob)  (phone-id  phone2 ) ) 

(draw-pixels  (id  picture)  (pixels  blob)  shade  offset 
(phone-id  blob)  (phone-id  phone2 ) ) ) ) ) 

(defmethod  draw-pixels 
(picture-id  pixel-list  shade  offset  phone-id  phone2-id) 
(dolist  (pixel  pixel-list  t) 
(let  ((time  (*  *pixel-size*  (first  pixel))) 
(freq  (*  *pixel-size*  (second  pixel ))) ) 
(if  (equal  phone-id  phone2-id) 
(setf  freq  (+  *offset*  freq))) 
(setf  freq  (+  freq  offset)) 
(if  (=  shade  1) 

(cw:draw-f illed-rectangle-xy  picture-id  time  freq 
*pixel-size*  *pixel-size*  :color  cw:black) 
(if  (=  shade  2) 

(cw :draw- f illed-rectangle-xy  picture-id  time  freq 
*pixel-size*  *pixel-size*  :color  cw:black) 
(if  (=  shade  3) 

(cw:draw-f illed-rectangle-xy  picture-id  time  freq 
*pixel-size*  *pixel-size*  :color  cw:red) 
(if  (=  shade  4) 

(cw:draw-f illed-rectangle-xy  picture-id  time  freq 
*pixel-size*  *pixel-size*  :color  cw:blue) 
(cw:draw-f illed-rectangle-xy  picture-id  time  freq 
*pixel-size*  *pixel-size* ) ))))))) 

(defmethod  identify-region  (picture- id  region  offset  phone-id  phone2-id) 
(let  ((x  (*  *pixel-size*  (time-max  (eval  region)))) 
(y  (*  *pixel-size*  (freq-max  (eval  region))))) 
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(if  (equal  (blackboard- id  (eval  region))  'lower) 

(setf  offset  *low*) 

(setf  offset  *high*) ) 

(setf  y  (+  y  offset ) ) 

(if  (equal  phone-id  phone2-id) 

(setf  y  (+  y  *of fset*) ) ) 

(setf  (cw:window-stream-x-position  picture-id)  x  ) 

(setf  (cw:window-stream-y-position  picture-id)  (+  y  4)) 

(format  picture-id  "~a"  (region-type  (eval  region))))) 

(defmethod  connect-matched-regions-r  (region-matches  (picture  screen)  phonel-id) 
(when  region-matches 

(connect-matched-region  (first  region-matches)  picture  phonel-id) 
(connect-matched-regions-r  (rest  region-matches)  picture  phonel-id))) 

(defmethod  connect-matched-region  (region-match  (picture  screen)  phonel-id) 

(let*  ( (regionl  (first  region-match))  (region2  (second  region-match)) 

(pixell  (middle  (pixels  (eval  regionl)))) 

(pixel2  (middle  (pixels  (eval  region2)))) 

(xl  (*  *pixel-size*  (first  pixell))) 

(x2  (*  *pixel-size*  (first  pixel2))) 

(yl  (*  *pixel-size*  (second  pixell))) 

(y2  (*  *pixel-size*  (second  pixel2 )) )  offset) 

(if  (equal  (blackboard-id  (eval  regionl))  'lower) 

(setf  offset  *low*) 

(setf  offset  *high*) ) 

(setf  yl  (+  yl  offset) ) 

(setf  y2  (+  y2  offset) ) 

(if  (equal  (phone-id  (eval  regionl))  phonel-id) 

(cw:draw-line-xy  (id  picture)  xl  yl  x2  (+  "offset*  y2 ) ) 

(cw:draw-line-xy  (id  picture)  xl  (+  *offset*  yl)  x2  y2)) 

(if  (equal  (phone-id  (eval  regionl))  phonel-id) 

(setf  (cw:window-stream-x-position  (id  picture))  x2 ) 

(setf  (cw:window-stream-x-position  (id  picture))  xl ) ) 

(setf  (cw:window-stream-y-position  (id  picture)) 

(set-y-text-position  picture  regionl)) 

(format  (id  picture)  "~d"  (time-min  (eval  regionl))))) 
;  (format  (id  picture)  "~3,2f"  (third  region-match)))) 

(defmethod  set-y-text-position  ( (picture  screen)  region-name) 
(if  (equal  (blackboard-id  (eval  region-name))  'lower) 
(if  (  >  (lower-y-text-position  picture)  40) 
(setf  (lower-y-text-position  picture)  2) 
(setf  (lower-y-text-position  picture) 
(+  (lower-y-text-position  picture)  15))) 
(if  (  >  (upper-y-text-position  picture) 
(+  *high*  (+  55  (*  *pixel-size*  128)))) 

(setf  (upper-y-text-position  picture)  (+  *high*  15  (*  *pixel-size*  128))) 
(setf  (upper-y-text-position  picture) 
(+  (upper-y-text-position  picture)  15))))) 

(defmethod  draw-match-list  (matched-list 
(blackboardl  blackboard)  (blackboard2  blackboard) (picture  screen) ) 
(dolist  (matched-list-1  matched-list  nil) 

(let  ( (matched-pair-1  (first  matched-list-1))  pixell  xl  yl) 
(if  (equal  (blackboard-id  (eval  (first  matched-pair-1)))  'lower) 
(if  (equal  (screen-position  blackboardl)  'lower) 
(if  (equal  (phone-id  (eval  (first  matched-pair-1))) 
(phone- id  (phonel  blackboardl))) 
(let  () 

(setf  pixell  (middle  (pixels  (eval  (second  matched-pair-1 )))) ) 
(setf  xl  (*  *pixel-size*  (first  pixell))) 

(setf  yl  (+  (*  *pixel-size*  (second  pixell ) )  *low*  *offset*))) 
(let  () 

(setf  pixell  (middle  (pixels  (eval  (first  matched-pair-1))))) 
(setf  xl  (*  *pixel-size*  (first  pixell))) 
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setf  yl  (+  (*  *pixel-size*  (second  pixell))  *low*  *of f set* ) )  )  ) 
if  (equal  (phone-id  (eval  (first  matched-pair- 1 ) ) ) 
phone-id  (phonel  blackboard2 ) ) ) 
let  () 

setf  pixell  (middle  (pixels  (eval  (second  matched-pair-1) )))  ) 
setf  xl  (*  *pixel-size*  (first  pixell))) 

setf  yl  (+  (*  *pixel-size*  (second  pixell))  *low*  *offset*))) 
let  () 

setf  pixell  (middle  (pixels  (eval  (first  matched-pair-1))))) 
setf  xl  (*  *pixel-size*  (first  pixell))) 

setf  yl  (+  (*  *pixel-size*  (second  pixell ) )  *low*  *of f set* ) ) ) ) ) 
let  () 

if  (equal  (phone-id  (eval  (first  matched-pair-1))) 
phone-id  (phonel  blackboardl))) 
let  () 

setf  pixell  (middle  (pixels  (eval  (first  matched-pair-1))))) 
setf  xl  (*  *pixel-size*  (first  pixell))) 
setf  yl  (+  (*  *pixel-size*  (second  pixell))  *high*))) 
let  () 

setf  pixell  (middle  (pixels  (eval  (second  matched-pair-1 )))) ) 
setf  xl  (*  *pixel-size*  (first  pixell))) 
setf  yl  (+  (*  *pixel-size*  (second  pixell))  *high*)))) 
if  (equal  (phone-id  (eval  (first  matched-pair-1))) 
phone-id  (phonel  blackboard2 ) ) ) 
let  () 

setf  pixell  (middle  (pixels  (eval  (first  matched-pair-1))))) 
setf  xl  (*  *pixel-size*  (first  pixell))) 
setf  yl  (+  (*  *pixel-size*  (second  pixell))  *high*))) 
let  () 

setf  pixell  (middle  (pixels  (eval  (second  matched-pai r-1 )))) ) 
setf  xl  (*  *pixel-size*  (first  pixell))) 
setf  yl  (+  (*  *pixel-size*  (second  pixell ) )  *high*)))))) 
dolist  (matched-list-2  (rest  matched-list-1 )  nil) 
dolist  (matched-pair-2  matched-list-2  nil) 
(let  ((matched-pair-2  (first  matched-list-2))) 
when  (>  (first  (first  matched-pair-2))  0.5) 
let*  ( (pixel2  (middle 

pixels  (eval  (first  (second  matched-pair-2) ) ) ) ) ) 
x2  (*  *pixel-size*  (first  pixel2))) 
y2  (*  *pixel-size*  (second  pixel2)))) 

if  (equal  (blackboard-id  (eval  (first  (second  matched-pai r-2 ))) ) 
lower) 

setf  y2  ( +  y2  *low* ) ) 
setf  y2  (+  y2  *high*) ) ) 

if  (equal  (blackboard-id  (eval  (first  (second  matched-pair-2 ))) ) 
lower) 

setf  y2  (+  y2  *offset*))) 
cw:draw-line-xy  (id  picture)  xl  yl  x2  y2) ))))))) 

defmethod  draw-best-match-list  (matched-list 
blackboardl  blackboard)  (blackboard2  blackboard) (picture  screen) 
dolist  (matched-list-1  matched-list  nil) 

let  ((matched-pair-1  (first  matched-list-1))  pixell  xl  yl) 
if  (equal  (blackboard-id  (eval  (first  matched-pair-1)))  'lower) 
if  (equal  (screen-position  blackboardl)  'lower) 
if  (equal  (phone-id  (eval  (first  matched-pair-1))) 
phone-id  (phonel  blackboardl))) 
let  () 

setf  pixell  (middle  (pixels  (eval  (second  matched-pai r-1 )))) ) 
setf  xl  (*  *pixel-size*  (first  pixell))) 

setf  yl  (+  (*  *pixel-size*  (second  pixell))  *low*  *offset*))) 
let  () 

setf  pixell  (middle  (pixels  (eval  (first  matched-pair-1))))) 
setf  xl  (*  *pixel-size*  (first  pixell))) 

setf  yl  (+  (*  *pixel-size*  (second  pixell ) )  *low*  *offset*)))) 
if  (equal  (phone-id  (eval  (first  matched-pair-1))) 
phone-id  (phonel  blackboard2 ) ) ) 
let  () 


9X 


setf  pixell  (middle  (pixels  (eval  (second  matched-pair- 1 )))) ) 

setf  xl  (*  *pixel-size*  (first  pixell))) 

setf  yl  (+  (*  *pixel-size*  (second  pixell))  *low*  *offset*))) 

let  () 

setf  pixell  (middle  (pixels  (eval  (first  matched-pair-1 ) ) ) ) ) 

setf  xl  (*  *pixel-size*  (first  pixell))) 

setf  yl  (+  (*  *pixel-size*  (second  pixell))  *low*  *of f set* ) ) ) ) ) 

let  () 

if  (equal  (phone-id  (eval  (first  matched-pair-1))) 

phone-id  (phonel  blackboardl ) ) ) 

let  () 

setf  pixell  (middle  (pixels  (eval  (first  matched-pair-1))))) 

setf  xl  (*  *pixel-size*  (first  pixell))) 

setf  yl  (+  (*  *pixel-size*  (second  pixell))  *high*))) 

let  () 

setf  pixell  (middle  (pixels  (eval  (second  matched-pair-1 )))) ) 

setf  xl  (*  *pixel-size*  (first  pixell))) 

setf  yl  (+  (*  *pixel-size*  (second  pixell))  *high*)))) 

if  (equal  (phone-id  (eval  (first  matched-pair-1))) 

phone-id  (phonel  blackboard2) ) ) 

let  () 

setf  pixell  (middle  (pixels  (eval  (first  matched-pair-1))))) 

setf  xl  (*  *pixel-size*  (first  pixell))) 

setf  yl  (+  (*  *pixel-size*  (second  pixell))  *high*))) 

let  () 

setf  pixell  (middle  (pixels  (eval  (second  matched-pai r- 1 )))) ) 

setf  xl  (*  *pixel-size*  (first  pixell))) 

setf  yl  (+  (*  *pixel-size*  (second  pixell ) )  *high*)))))) 

(dolist  (matched-list-2  (rest  matched-list- 1 )  nil) 

(dolist  (matched-pair-2  matched-list-2  nil) 

let  ((matched-pair-2  (second  matched-list-1 )) ) 

(when  (>  (first  (first  matched-pair-2))  0.5) 

let*  ( (pixel2  (middle 

pixels  (eval  (first  matched-pair-2))))) 

x2  (*  *pixel-size*  (first  pixel2))) 

y2  (*  *pixel-size*  (second  pixel2 ))) ) 

if  (equal  (blackboard-id  (eval  (first  matched-pair-2))) 

lower) 

setf  y2  (+  y2  *low*) ) 

setf  y2  (+  y2  *high*) ) ) 

if  (equal  (blackboard-id  (eval  (first  matched-pair-2))) 

lower) 

setf  y2  (+  y2  *offset*) ) ) 

cw:draw-line-xy  (id  picture)  xl  yl  x2  y2) ) ) ) ) ) 


(defccnstant  SLICES  20) 
(defconstant  *  far- threshold*  0.15) 

(defmethod  match-far-phones 

( (blackboardl  blackboard)  (blackboard2  blackboard)  (picture  screen) ) 

(let  ( (match-listl  nil)  (match-list2  nil)  (best-match-list  nil) 

(hough-list  nil)  (new-match-listl  nil)  (new-match-list2  nil) 

(allowable- time-difference 

(*  (/  (abs  (-  (latitude  (phonel  blackboardl)) 

(latitude  (phone2  blackboard2 ) ) ) )  0.81)  1.2))) 

(dolist  (matched-pair  (definite-region-matches  blackboardl)  nil) 

(setf  match-listl 

(append  (list  (list  matched-pair 

(match- far- regions  allowable- time-difference 
matched-pair  blackboard2 ) ) ) 
match-listl ) ) ) 

(dolist  (matched-pair  (definite- region-matches  blackboard2)  nil) 

(setf  match-list2 

(append  (list  (list  matched-pair 

(match- far- regions  allowable- time-difference 
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matched-pair  blackboardl ) ) ) 

match-list2 ) ) ) 

(print  "MATCH-LIST1")  (print  match-listl) 
(print  "MATCH-LIST2")  (print  match-list2) 
(setf  hough-list  (get-hough-transf orm  allowable-time-dif ference 

match-listl  match-list2 ) ) 
(setf  new-match-listl  (apply-hough-list  allowable-time-difference  1 

hough-list  match-listl)) 

; (print  "MATCH-LIST1  after  HOUGH")  (print  new-match-listl) 

(setf  new-match-list2  (apply-hough-list  allowable-time-difference  -1 

hough-list  match-list2) ) 

; (print  "MATCH-LIST2  after  HOUGH")  (print  new-match-list2 ) 

(setf  best-match-list  (get-best-matches  new-match-listl  new-match-list2 ) ) 

(draw-best-match-list  best-match-list  blackboardl  blackboard2  picture) 

(dolist  (matched-pair  best-match-list  nil) 

(setf  match-listl  (remove-main-match  (first  matched-pair)  match-listl)) 

(setf  match-list2  (remove-main-match  (second  matched-pair)  match-list2 ) ) ) 

(dolist  (matched-pair  best-match-list  nil) 

(setf  match-list2  (remove-matched-pair  (first  matched-pair)  match-list2) ) 

(setf  match-listl  (remove-matched-pair  (second  matched-pair)  match-listl)) 

(print  "MATCH-LIST1  AFTER  REMOVAL")  (print  match-listl) 

(print  "MATCH-LIST2  AFTER  REMOVAL")  (print  match-list2) 

(setf  hough-list  (get-hough-transform  allowable-time-difference 

match-listl  match-list2 ) ) 
(setf  new-match-listl  (apply-hough-list  allowable-time-difference  1 

hough-list  match-listl)) 

(setf  new-match-list2  (apply-hough-list  allowable- time-difference  -1 

hough-list  match-list2 ) ) 

(setf  best-match-list  (append  best-match-list 
(get-best-matches  new-match-listl  new-match- 1 ist2 )) ) 

(draw-best-match-list  best-match-list  blackboardl  blackboard2  picture) 
(print  "BEST-MATCH-LIST")  (print  best-match-list) 
(dolist  (matched-pair  best-match-list  nil) 

(setf  match-listl  (remove-main-match  (first  matched-pair)  match-listl)) 
(setf  match-list2  (remove-main-match  (second  matched-pair)  match-list2 ) ) ) 
(dolist  (matched-pair  best-match-list  nil) 

(setf  match-list2  (remove-matched-pair  (first  matched-pair)  match-list2 ) ) 
(setf  match-listl  (remove-matched-pair  (second  matched-pair)  match-listl)) 
(print  "MATCH-LIST1  AFTER  SECOND  REMOVAL")  (print  match-listl) 
(print  "MATCH-LIST2  AFTER  SECOND  REMOVAL")  (print  match- list2) 
)  ) 
;  (draw-match-list  match-listl  blackboardl  blackboard2  picture))) 

(defmethod  match-far-regions 
(allowable-time-difference  matched-pair-1  ( the-blackboard  blackboard)) 
(let  ((region-1  (if  (>  (area  (eval  (first  matched-pair-1))) 
(area  (eval  (second  matched-pair-1 ))) ) 
(first  matched-pair-1) 
(second  matched-pair-1) ) ) 
( list-of-matches  nil)) 

(dolist  (matched-pair-2  (definite-region-matches  the-blackboard)  nil) 
(let*  ( (region-2  (if  (>  (area  (eval  (first  matched-pair-2))) 
(area  (eval  (second  matched-pair-2 ))) ) 
(first  matched-pair-2) 
(second  matched-pair-2) ) ) 
(match- factor  (match- region- factor 
region-1  region-2  allowable- time-difference) ) ) 
(if  (>  (first  match-factor)  0) 
(setf  list-of-matches 

(append  (list  (list  match-factor  matched-pair-2)) 
list-of-matches) ) ) ) ) 
(if  list-of-matches 
(bubble-sort  list-of-matches) ) ) ) 

;  The  Common  Lisp  Companion  pg .  37  8 
(defmethod  bubble-sort  (input-list) 

(let  (done) 

( loop 
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(setf  done  t) 

(do  ((to-do  input-list  (rest  to-do))) 

((equal  (length  to-do)  1)) 

(when  (<  (first  (first  (first  to-do))) 

(first  (first  (second  to-do)))) 

(swap -elements  0  1  to-do) 

(setf  done  nil ) ) ) 

(if  done  (return  input-list))))) 

The  Common  Lisp  Companion  pg .  376 
[defmethod  swap-elements  (n  m  input-list) 
(let  ((nth-value  (nth  n  input-list))) 
(setf  (nth  n  input-list)  (nth  m  input-list)) 
(setf  (nth  m  input-list)  nth-value) 
input-list) ) 

; defmethod  get-hough-transform 

allowable-time-difference  match-listl  match-list2) 

let  ((hough-list  nil)  (minimum  1000)  (maximum  0) 

time-slice  (*  (/  allowable-time-difference  SLICES)  2.0)) 

offset  (/  SLICES  2.0) ) ) 

dotimes  (index  SLICES  nil)  (setf  hough-list  (append  '(0)  hough-list))) 

dolist  (best-match-list  match-listl  nil) 

if  (first  (second  best-match-list)) 

if  (>  (first  (first  (first  (second  best-match-list ))) )  0.5) 

let  ((index  (floor  (+  offset 

/  (second  (first  (first  (second  best-match-list)))) 
time-slice) ) ) ) ) 

if  (<  index  0)  (setf  index  0)) 

setf  (nth  index  hough-list) 

+  (first  (first  (first  (second  best-match-list)))) 

nth  index  hough-list) ) ) ) ) ) 

if  (second  (second  best-match-list) ) 

if  (>  (first  (first  (second  (second  best-match-list))))  0.5) 

let  ((index  (floor  (+  offset 

/  (second  (first  (second  (second  best-match-list)))) 
time-slice) ) ) ) ) 

if  (<  index  0)  (setf  index  0)) 

setf  (nth  index  hough-list) 

+  (first  (first  (second  (second  best-match-list)))) 

nth  index  hough-list) ) ) ) ) ) 

if  (third  (second  best-match-list)) 

if  (>  (first  (first  (third  (second  best-match-list))))  0.5) 

let  ((index  (floor  (+  offset 

/  (second  (first  (third  (second  best-match-list)))) 
time-slice) ) ) ) ) 

if  (<  index  0)  (setf  index  0)) 

setf  (nth  index  hough-list) 

+  (first  (first  (third  (second  best-match-list)))) 

nth  index  hough-list) )))))) 

dolist  (best-match-list  match-list2  nil) 

if  (first  (second  best-match-list)) 

if  (>  (first  (first  (first  (second  best-match-list))))  0.5) 

let  ((index  (floor  (+  offset  (*  -1.0 

/  (second  (first  (first  (second  best-match-list)))) 

ime-slice) ) ) ) ) ) 

if  (<  index  0)  (setf  index  0)) 

setf  (nth  index  hough-list) 

+  (first  (first  (first  (second  best-match- list ))) ) 

nth  index  hough-list) ) ) ) ) ) 

if  (second  (second  best-match-list)) 

if  (>  (first  (first  (second  (second  best-match-list))))  0.5) 

let  ((index  (floor  (+  offset  (*  -1.0 

/  (second  (first  (second  (second  best-match-list)))) 

ime-slice) ) ) ) ) ) 

if  (<  index  0)  (setf  index  0)) 

setf  (nth  index  hough-list) 

+  (first  (first  (second  (second  best-match-list ))) ) 
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nth  index  hough-list) ) ) ) ) ) 

if  (third  (second  best-match-list)) 

if  (>  (first  (first  (third  (second  best-match- list ))) )  0.5) 

let  ((index  (floor  (+  offset  (*  -1.0 

/  (second  (first  (third  (second  best-match-list)))) 
time- si  ice) ) )  ) ) ) 

if  (<  index  0)  (setf  index  0)) 

setf  (nth  index  hough-list) 

+  (first  (first  (third  (second  best-match-list)))) 

nth  index  hough-list) )))))) 

dolist  (element  hough-list  nil) 

if  (<  element  minimum)  (setf  minimum  element))) 

setf  hough-list  (mapcar  #' (lambda  (x)  (-  x  minimum))  hough-list)) 

setf  hough-list  (mapcar  #' (lambda  (x)  (sqrt  x) )  hough-list)) 

dolist  (element  hough-list  nil) 

if  (>  element  maximum)  (setf  maximum  element))) 

if  (>  maximum  0.0) 

setf  hough-list  (mapcar  #' (lambda  (x)  (double- float 

/  x  maximum) ) )  hough-list) ) ) 

print  "HOUGH-LIST") 

print  hough-list) 
hough-list) ) 

(defmethod  apply-hough-1 ist 
(allowable-time-difference  direction  hough-list  match-list) 
(let  ((time-slice  (*  (/  allowable-time-difference  SLICES)  2.0)) 
(offset  (/  SLICES  2.0))  (new-match-list  nil)) 
(dolist  (sub-match-list  match-list  new-match-list) 
(setf  new-match-list 
(append  new-match-list 
(list  (list  (first  sub-match-list) 
(apply-hough- list2  (rest  sub-match-list) 
direction  time-slice  offset  hough-list) ))))))) 

(defmethod  apply-hough-list2 
(sub-sub-match-list  direction  time-slice  offset  hough-list) 
(let  ( (new-list  nil) ) 

(dolist  (match-element  (first  sub-sub-match-list)  new-list) 
(setf  new-list 

(append  new-list  (list  (list  (apply-hough-list3  (first  match-element) 
direction  time-slice  offset  hough-list) 
(second  match-element) ) ) ) ) ) 
(if  new-list  (setf  new-list  (bubble-sort  new- list )))) ) 

(defmethod  apply-hough-1 ist3 
(list-element  direction  time-slice  offset  hough-list) 
(let  ({index  (floor  (+  offset  (*  direction 
(/  (second  list-element)  time-slice)))))) 
(if  (>=  index  SLICES)  (setf  index  (1-  SLICES))) 
(if  (<  index  0)  (setf  index  0)) 

(list  (*  (first  list-element)  (nth  index  hough-list)) 
(second  list-element) ) ) ) 

(defmethod  get-best-matches  (match-listl  match-list2) 
(let  ((best-match-list  nil)  match-list-1  match-list-2 ) 
(setf  match-list-1  match-listl) 
(setf  match-list-2  match-list2) 
(dolist  (matched-pair-1  match-list-1  nil) 
(when  (match-equal-p  (first  matched-pair-1) 
(get-best-match  (second  (first  (second  matched-pair-1 )) ) 
match-list-2 ) ) 
(print  "best  value  =") 

(print  (float  (first  (first  (first  (second  matched-pair-1 )))))  ) 
(if  (>  (first  (first  (first  (second  matched-pair-1 ))) ) 
*far-threshold* ) 
(setf  best-match-list 

(append  (list  (list  (first  matched-pair-1) 
(second  (first  (second  matched-pair-1 )))) ) 
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best-match-list) ) ) ) ) 
best-match- list ) ) 

(defmethod  match-equal-p  (matched-pair-1  matched-pair-2 ) 
(if  (equal  (first  matched-pair-1)  (first  matched-pair-2)) 
t 

nil)  ) 

(defmethod  get-best-match  (matched-pair-1  match-list) 
(when  match-list 

(if  (match-equal-p  matched-pair-1  (first  (first  match- list )) ) 
(let  () 

(when  (second  (first  match-list)) 
(second  (first  (second  (first  match-list)))))) 
(get-best-match  matched-pair-1  (rest  match-list))))) 

(defmethod  remove-matched-pair  (target-matched-pair  match-list) 
(if  match-list 

(append  (list  ( remove-matched-pair-subl 
target-matched-pair  (first  match-list))) 
(remove-matched-pair  target-matched-pair  (rest  match-list))))) 

(defmethod  remove-matched-pair-subl  (target-matched-pair  sub-listl) 
(append  (list  (first  sub-listl)) 
(list  ( remove-matched-pair-sub2  target-matched-pair  (second  sub-listl)))) 

(defmethod  remove-matched-pair-sub2  (target-matched-pair  sub-list2) 
(if  (first  sub-list2) 

(if  (match-equal-p  target-matched-pair  (second  (first  sub-list2))) 
(rest  sub-list2) 

(append  (list  (first  sub-list2)) 
( remove-matched-pair-sub2  target-matched-pair  (rest  sub-list2 ) ) ) ) ) ) 

(defmethod  remove-main-match  (matched-pair  match-list) 
(if  match-list 

(if  (match-equal-p  matched-pair  (first  (first  match- list )) ) 
(rest  match-list) 
(append  (list  (first  match-list)) 
(remove-main-match  matched-pair  (rest  match- list ))))) ) 
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